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We recently mailed our annual Readers’ Survey to a subset of subscribers. If 
you received one, I encourage you to fill it out and return it to us. We value the 
feedback from readers. On that note, we received some comments in response to 
the coverage of LDAP in our May issue and, because of space constraints this 
month, I’m excerpting one of the letters here. Alf Wachsmann wrote: 


While reading the two articles about LDAP in your May 2004 issue of the Sys Admin 
magazine I became rather angry: LDAP can NOT be used securely as an authentica- 
tion system! If someone is looking for alternatives for their current system, look at 
Kerberos 5 (MIT or Heimdal). 


Also, NIS is not insecure per se. It is insecure as an authentication system but has no 
problems with security as an authorization system (use “securenets” and block NIS 
ports on your firewall). 


The real shortcoming of LDAP when used for authentication is the fact that encrypted 
passwords travel over the wire. Modern authentication systems, like Kerberos 5, do not do 
this. LDAP also cannot provide mutual authentication mechanisms for services. Kerberos 
has many more built-in security features like a replay cache or pre-authentication. 


Hal Pomeranz, our technical editor, responded as follows: 


Given the way the articles turned out, your criticisms are generally well founded. We 
had hoped to include information on SASL binding between LDAP clients and 
servers, but that did not happen. Once the client-server communication is fully 
encrypted and there’s strong authentication between the LDAP client and server, I 
actually consider LDAP to be a reasonably secure (though not perfect) authentication 
system, but unfortunately that was not the full scenario presented in the articles. We 
hope to acquire a follow-up piece to complete the picture. 


That being said, I’m also a big fan of Kerberos for authentication. What’s always been 
interesting to me has been the perception of Kerberos as “hard” to do, and yet sites 
seem to be willing to invest as much or more effort in converting over to LDAP-based 
systems for authentication. I suspect this is because there are a lot of “canned” NIS to 
LDAP migration tools out there, so people perceive that the transition is smoother. Or 
it could be the lack of a clear general naming services choice for Kerberos (whither 
Hesiod?) — people know that they need to store more than just basic authentication 
information, so LDAP seems attractive as a “single source” for all of their network 
data. Again, I think it’s a perception more than reality perhaps. 


I'd frankly be interested in more Kerberos-related content in Sys Admin magazine. In 
fact I'd dearly love somebody to write an article describing how to authenticate Unix 
Kerberos clients to Active Directory servers and have a single login for users across 
the Unix and Windows domains. 


If you’re interested in writing an article for Sys Admin magazine on Kerberos, 
LDAP, or some other topic, or simply have feedback to share, please send your 
proposals to Rikki Endsley (rendsley@cmp.com) and your comments to me 
(aankerholz@cmp.com). We look forward to hearing from you. 


Sincerely yours, 


Amber Ankerholz 
Editor in Chief 
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TKCluster 


Tom Kunz 


inux has shown a lot of growth in the area of data-centric, 

high-availability clustering. Most admins are already 

familiar with computational clusters, known loosely as 
Beowulf clusters, which are implemented in the form of MPI, 
PVM, LAM, MOSIX, and other process-sharing and process-dis- 
tributing technologies. There are also “Web service clusters”, such 
as those distributed in years past by TurboLinux and others. These 
were typically groups of similarly configured servers that used DNS 
and round-robin IP address tricks to give the illusion of Web server 
high-availability to end users. 

Cohesive operation between the 
nodes, however, was still only 
achieved through a shared-storage 
medium, such as Fibre Channel or 
shared SCSI, which are prohibitively 
expensive for small businesses, or 
proprietary cluster hardware and soft- 
ware, which is also prohibitively 
expensive. A database engine that 
serves a Web cluster must still itself 
be clustered to achieve true high- 
availability. Application-level high- 
availability tools (such as the MySQL 
database engine) that transparently 
replicate themselves between servers, 
are also being used to provide some 
level of redundancy. 

The one area in which Linux still 
starves for attention is in the realm 
of lightweight, easily configured, affordable high-availability — a 
general-purpose cluster. A general-purpose, high availability cluster 
must be “application agnostic” — it should not care what runs on it, 
whether it be Web server, mail server, database server, or any future, 
yet-unknown type of service. The cluster should give a uniform 
style of operation no matter what application is running. 

In response to this, I have written TKCluster (when I initially 
wrote it, I couldn’t think of a good name for it, so I just prefixed 
“cluster” with my initials). TKCluster is available under the 
GPL so that anyone can freely download and modify it to suit 
their needs. 


Overview 

TKCluster itself is a cluster manager. Raw data replication 
between nodes is performed by the wonderful DRBD driver by 
Phillipp Reisner. DRBD is a block device that maps to a given 
raw disk partition and a socket. Writes to the DRBD device 
(/dev/nb0O .. /dev/nbX) go to both the physical disk in the local 
machine, as well as to the waiting secondary node over a standard 
Ethernet connection. All clusters require some kind of “heartbeat” 
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mechanism. After experimenting with various ones, I chose 
openMosix. openMosix was designed to share computation-inten- 
sive process loads between multiple machines; however, I have yet 
to find anything that does as good a job at maintaining a frequently 
updated list of connected machines. 

The process-load sharing and MFS filesystem (analogous to tra- 
ditional NFS, but infinitely smarter) make openMosix a perfect 
candidate for helping to tie the cluster nodes together. Although 
MES is not necessary for TKCluster operation, it sure helps when 
copying configuration files around 
between the nodes of the cluster. 
TKCluster’s role is to use the data 
gleaned from both openMosix and 
DRBD to make decisions about 
starting services, seizing control of 
the cluster IP address, and keeping 
the sister copy of TKCluster on the 
other node of the cluster aware of 
what’s going on. 


Design 

TKCluster is intended for use in 
clusters where data is to be shared 
between two nodes, a primary and a 
secondary. The data that is “shared” 
between them lives on one or more 
partitions, each partition having its 
own DRBD device. These partitions 
should be separate from the system 
partitions (/, /boot, /usr, /var, etc.), because the secondary node will 
have no access to these partitions until the primary dies (and the 
secondary seizes control) or the primary gives up control of the par- 
tition. The secondary simply waits for something to happen and 
then acts accordingly to take control of the partition and restart the 
services previously served by the primary. 

These DRBD devices are “raw” devices, they need not have a 
filesystem on them. Database engines such as Oracle or Informix 
can be configured to use “raw” partitions. Because DRBD is a block 
device driver, it simply passes raw block writes through to the local 
disk. The application running on top of it has no concern about the 
underlying device, as long as requests are satisfied and the driver 
behaves as a block device driver should. A DRBD device can also 
contain a traditional filesystem such as ext3, and be mounted in the 
usual fashion. DRBD allows the data sharing to be thoroughly 
agnostic of the apps that talk to it. 

When the cluster is fully configured properly and initially pow- 
ered up, the primary will talk to the secondary and push its data over 
to the secondary as needed. DRBD has the capability to do a full 
synchronization, meaning a direct, block-for-block copy of the entire 
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Sophos PureMessage protects more than 2,000 email users 
for Coast Capital Savings Credit Union 


Customer: Coast Capital Savings Credit Union 
Industry: Banking 
Product: Sophos PureMessage v4.5 


Business Overview 

Coast Capital Savings Credit Union is Canada’s second largest 
credit union with $6.4 billion in assets, 300,000 members and 42 
branches across the Fraser Valley, Lower Mainland and 
Vancouver Island. 


Business Challenge 

Like many banking organizations, Coast Capital experienced an 
influx of spam and needed to ensure that their network was 
protected at all times against the threat of viruses. Having a 
scalable message processing and quarantine architecture, and the 
ability to customize email policies, was a critical part of the 
decision making process. 


Coast Capital’s systems include both Microsoft Exchange servers 
and UNIX-based servers to relay and filter messages. Managing 
and enhancing network services relating to the company’s 
various systems, including email communications and virus 
security, Andrew Banman understood that selecting and 
implementing the right integrated security solution would be a 
challenging task. 


“Support, functionality, and customization were key factors in 
selecting Sophos PureMessage,” says Banman. “The quarantine 
digests diminished false positive issues and was the most 
significant feature that influenced our selection of PureMessage.” 


The PureMessage Solution 

According to Banman, “After selecting Sophos PureMessage, 
we’ve been able to filter a great deal of spam, which saves untold 
man-hours, but even more critical is the virus checking. By 
catching email-borne viruses at the front door, we mitigate a 
great deal of risk.” 


“Over the past 2 years, Sophos’s support team has helped me to 
better understand the PureMessage product so that I can tune it 
for my company’s specific needs. The one time that I 
experienced an issue, tech support responded immediately and 
made it a priority to resolve the issue.” 


“Unlike other anti-spam vendors, which make customization 
more complex and ultimately slows the process, Sophos 
PureMessage was a breeze to install,” says Banman. “Installing 
PureMessage was very straightforward and much of the required 
functionality was there by default.” 


SOPHOS 


Coast Capital tested PureMessage with a small group for several 
months to ensure it was operating as desired before rolling it out 
company-wide. “During the test phase, more and more users 
requested to be added to the test group,” says Banman. “In fact, 
the growing anticipation of the product forced us to move up 
the release date.” 


Sophos PureMessage now protects over 2,000 internal users. 
According to Coast Capital statistics, PureMessage handles 
between 13,000 and 50,000 messages each day. Of those figures, 
it is estimated that up to 50% is virus or spam related. 


Results 

When it comes to detecting and protecting against viruses, 
Sophos PureMessage receives top rating from Coast Capital 
Savings. “We have only used one other mail virus filtering 
application prior to PureMessage,” says Banman. “In our case, we 
feel that PureMessage not only offers better detection but 
superior capabilities.” 


Inbound mail is processed with the standard PureMessage spam 
and virus rules plus a few custom ones that have been set up. 
The organization’s outbound mail is currently checked only for 
viruses. In the test and implementation phase, Sophos developed 
a custom Milter add-on to conduct LDAP lookups against Coast 
Capital’s Active Directory that includes a corporate signature. 
Having policy management in place helps organizations meet 
security, communication and regulatory compliance needs. 


Of course, with any new implementation, there are always 
challenges. According to Banman, there were two. “Customizing 
the policies requires some focused time. However, one of the 
greatest challenges is often managing user expectations.” 


“We are very pleased with the product,” says Banman. “Overall, 
end users at Coast Capital have reacted very positively.” 


More Information 
PureMessage is a complete email filtering system for spam and 
virus protection, and corporate policy enforcement. 


Contact Sophos 


For FREE anti-spam white papers, visit: 
www.sophos.com/link/sysO 704 


SOPHOS INC 


Eastern and Central USA: 
Tel 1 781 973 0110 

Toll free 1 888 767 4679 
Email salesus@sophos.com 


Western USA and Canada: 
Tel 1 604 484 6400 

Toll free 1 866 866 2802 
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partition, or of doing a “fast” synchronization, where it only copies 
over changes it finds. For the most part, a cold boot of the cluster will 
always result in a full sync, while momentary loss of connectivity 
between primary and secondary may result in “fast” syncs. 

Once the sync is done, the secondary’s partition will have all the 
data of the primary’s. All future writes on the primary will flow over 
to the secondary and be committed there as well. 

While up, openMosix will be active on both machines and will 
maintain its own list of what machines are currently up. The process- 
sharing capabilities of openMosix may be used, although most 
servers do not cause excessive computational loads. openMosix judi- 
ciously migrates processes that are strictly computational, and gener- 
ally keeps I/O-bound processes local to the machine on which they 
were started. The main purpose of openMosix will be to actively 
collect availability data so that the secondary node of the cluster can 
accurately and reliably detect when a primary node dies, and then 
grab the services of the primary accordingly. 

Because DRBD is intended for duplication of data between 
only two nodes, scaling the data high availability of TKCluster 
beyond two nodes is not possible. However, thanks to 
openMosix, if TKCluster is to be used to service a computational 
cluster, additional “openMosix-only” nodes can be added to the 
cluster. These nodes would not be running DRBD, and would 
only participate in the computationally intensive processes in the 
cluster. The computational horsepower of the cluster can be mul- 
tiplied as needed by adding more and more “openMosix-only” 
nodes to the cluster, although this is generally only needed in 
scientific and research-oriented endeavors. 


Hardware 

TKCluster is intended to enable small- to medium-sized busi- 
nesses to gain the benefits of data high availability without paying 
large sums for commercial clustering hardware and software. As 
such, this article will highlight the installation and usage of 
TKCluster as it is currently installed at one of my customer’s sites. 
The hardware used in this example is not the cheapest of the cheap, 
nor is it the most expensive. It is all fairly standard, commonly 
available PC hardware. 

If you choose to duplicate this particular installation today, 
depending on your hardware vendor, initial hardware outlay will be 
less than $2000. Most likely it will be substantially less. Even 
lower-powered machines are quite usable with TKCluster, however, 
my customer specifically chose this configuration because of previ- 
ous experience with the same hardware. J am a small business 
owner, and I have designed TKCluster so that others in a similar 
situation should be able to use it as well. 

Two machines, identically configured in hardware in BIOS, are 
preferred to make life easy on the administrator. It’s possible to have 
very different hardware and chipsets between the two nodes of the 
cluster, however, I strongly recommend identical hardware configura- 
tions. This will vastly simplify the administrative overhead if anything 
is ever changed inside the servers. The hardware selected for each 
machine for this configuration consists of the following: 


* Asus P4P800 motherboard (includes 3Com 3c940 gigabit LAN 
and Intel 865G/ICHS chipset) 

* 512MB Crucial PC-3200 RAM (2 x 256M DIMMs) 

¢ Western Digital 36GB SATA “Raptor” HD 

* Intel 2.6 GHz CPU, 800 MHz FSB 

* Intel Pro/1000 MT gigabit NIC 

¢ A generic case, power supply, video card, CD-ROM, and floppy 
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Hardware gurus familiar with Intel products will immediately notice 
that the motherboard and NIC are not traditionally considered 
“server-class”, because of the lack of the faster PCI-X architecture. 
However, in this configuration, specifically aiming to be affordable 
for the small business owner, it will become apparent as to why a 
PCI-X motherboard and PCI-X version of the Intel Pro/1000 gigabit 
card are not completely necessary. 

The limiting factor of this particular configuration is not neces- 
sarily the PCI bus. Major performance benefits will not be realized 
by simply adding PCI-X to the configuration. If you bring up one of 
the two nodes of the cluster with Fedora Core 1.0, the libata drivers 
from Fedora will address the SATA hard drive as /dev/sda. Once 
installed and running, you can run hdparm -Tt /dev/sda and get 
something like the following: 


# hdparm -Tt /dev/sda 
/dev/sda: 


Timing buffer-cache reads: 
Timing buffered disk reads: 


3076 MB in 2.00 seconds = 1538.00 MB/sec 
160 MB in 3.03 seconds = 52.81 MB/sec 


Repeated executions of hdparm will average something about 1500- 
1700 MB/sec cached, and 50-53 MB/sec uncached. The maximum 
sustained throughput of this particular disk is about 53 MB/sec. In 
the grand scheme of things, virtually all hard drives, whether ATA, 
SATA, or SCSI, have a maximum sustained throughput of between 
40 and 60 MB/sec, so this particular disk is in the upper-middle 
range of performance. There are other articles that analyze the 
performance throughput of various configurations, but without 
going up to a RAID configuration using much more expensive 
controller cards and multiple disks, the best single-disk through- 
put realizable with today’s hard-drive technology is a sustained 60 
MB/sec or so. For small business owners, where every penny 
counts, cost is a compelling reason to stay with a single disk on 
the built-in controller and not opt for a more expensive RAID con- 
troller connected to several disks. 

Some users may see the glossy advertisements of “320 MB/sec 
SCSI” and start thinking their systems simply must have it. 
However, remember that the advertised 320 MB/sec is not the sus- 
tained throughput; it is only the speed at which data moves from the 
host controller SCSI card to the disk’s electronics and into the cache 
on the disk itself. SATA disk caches are typically 8M and only the 
most expensive SCSI disks have 16M of cache onboard. The actual 
cache-to-platter speed is the important speed; it is that speed that 
limits sustained throughput to the disk. 

While this particular disk is physically limited to about 53 
MB/sec, the theoretical limit of a gigabit NIC, connected to the 
fastest possible PCI slot, is 125 MB/sec. The NIC in this is not the 
limiting factor for duplicating data between the nodes of the cluster; 
the disk is still the limiting factor. The practical limits of a NIC on a 
plain PCI bus (not PCI-X) are still only a bit faster than that of the 
hard drive itself. 

The choice of a cheaper motherboard and no RAID controller 
sounds logical in a theoretical sense — it seems to work out on paper 
that the limiting factor will be the hard drive itself. Increasing disk 
performance could be a major expense, which could prohibit small 
businesses from wanting to buy into an expensive pair of machines. 
But some are still likely to be skeptical, believing that maybe getting 
a “server-class” motherboard will boost performance substantially. 
If you’re skeptical, that’s fine, but I'll cut to the chase — cluster 
performance for data replication between the two cluster nodes is 
within a few percent of the disk’s maximum throughput. 
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With a maximum sustained disk throughput of 50-53 MB/s, this 
cluster configuration showed a consistent 47 to 50 MB/sec replica- 
tion speed. Factoring in IP latency, block-copy overhead, and data 
transfer across the PCI bus from disk to NIC, a few MB/sec perfor- 
mance hit isn’t so bad after all. In this example, the customer’s 
18GB DRBD partition replicated itself between the two servers in 
about 7 minutes. 

The only way to really boost the limiting factors would be a more 
expensive RAID card (not necessarily just getting a motherboard with 
one of the onboard RAID chipsets) connected to multiple disks. 
Since a good 3Ware card and several disks would cost about the 
same price as the rest of the machine, and a 
new motherboard would cost substantially 
more than the Asus P4P800, it is likely to 
be cost-prohibitive for small business own- 
ers to use that. However, for enterprise 
solutions with dozens or hundreds of users, 
the cost may be well justified. 

I have had opportunity to work with sev- 
eral flavors of gigabit NIC, but I keep coming 
back to the Intel Pro/1000 MT for both relia- 
bility and economy. It’s not that I’m an “Intel 
bigot” or something— I use DLink, NetGear, 
Realtek, and 3Com cards in a lot of 
machines, but not as the NIC on which all the 
cluster data rides. If you have found a partic- 
ular flavor of gigabit card that you trust, 
which has seen a lot of long-term sustained 
throughput without any hiccups, this cluster 
configuration will still work well for you. 


Software 

This cluster relies on several pieces of 
GPL software to function properly. The full 
list can be found in the Resources section, 
but the software is summarized here: 


¢ TKCluster — http://www.solidrock- 
technologies.com/clustermanager 
* openMosix — 
http: //www.openmosix.org 
¢ DRBD — http://www. ]inbit.com 
¢ Heartbeat — http://www. ]inux-ha.org 
¢ Various kernel patches 


The current version of TKCluster was devel- 
oped against the above packages, using 
Fedora Core 1.0 as the base distribution. 
TKCluster consists of Perl and shell scripts, 
so anyone familiar with those should be able 
to edit and alter it as necessary. 


myself) as well as performance weaknesses. Jeff Garzik has writ- 
ten a nice SATA driver for the ICH5 SATA as well as other VIA 
and SII chipsets. This comes as a set of kernel patches found at 
kernel.org: 


http://www. kernel .org/pub/linux/kernel/people/jgarzik/libata/ 


Unfortunately, the kernel patches don’t seem to be developed against 
the canonical kernel. Some of the diffs in the file 2.4.22-libatal .patch 
did not patch cleanly against 2.4.22. However, this is not a showstop- 
per by any means. You can manually patch the files that don’t patch. 
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sk98lin Drivers 

There’s a fair amount of “fiddling” that had to be done to get the 
kernel patched. I was enthralled with the amount of “extra” drivers 
for the gigabit adapters in the Fedora-distributed kernel and so I 
attempted to develop some patches against the Fedora-distributed 
2.4.22 kernel source. However, because it seems to use the new 
scheduler code, patches for openMosix did not apply cleanly to the 
Fedora-distributed source. In fact, the patches failed miserably 
against the scheduler code and resulted in unusable source. Also, 
because of the difficulties encountered with the libata patches, I was 
very disappointed not to be able to develop a concise patch for 
Fedora’s 2.4.22 source. 


openMosix 

I ended up pulling canonical kernel source for 2.4.22 straight 
from kernel.org, and then patching with openMosix. Once the 
openMosix patches were applied, I copied the gigabit drivers 
(notably the sk98lin driver for the 3Com 3c940 adapter built into the 
P4P800 motherboard) directly from the Fedora source, as well as 
the files for the libata source. I went through the libata patch file and 
copied all relevant code into the proper directories. I believe you 
will also find the latest e1000 driver patches in there, which work 
with a larger variety of Intel Pro/1000 variants and board revisions 
than the canonical drivers. If you find that your Pro/1000 is not 
automatically detected by the driver, try downloading the latest 
version of the driver directly from Intel and compiling it. New board 
revisions occur all the time, and the canonical kernel drivers 
become outdated quickly. 

If you are a bit squeamish about this level of patching, I have 
provided a reference page containing links to the kernel source that 
I patched. The kernel tarball available on that page contains only the 
canonical kernel source plus the aforementioned drivers that I 
copied in from the Fedora source. The ata_piix driver from Jeff 
Garzik is quite nice and offers tremendous performance. If you 
are using mod_scsi and, especially if you are using the driver to 
talk to a CD/DVD burner, pay special attention to that driver. If 
you accidentally use the “old” driver instead of the new one, your 
machine is likely to hang on boot when the module is inserted. 


Setup and Applications 


The easiest way to get the cluster up and running is the following: 


1. Assemble all hardware, but put both SATA hard drives into one of 
the machines. The second hard drive will be available to dupli- 
cate the first one onto it when the first is fully configured. 

2. Install Fedora Core 1.0 (2.0 is not available in non-beta at the 
time of this writing, although it probably will be by the time this 
goes to press). 

Make sure you get Perl installed because TKCluster is written 
mostly in Perl. During installation, set aside at least one large 
partition that will be used as the “cluster” partition. I set up this 
particular customer with the following disk layout: 


Disk /dev/sda: 37.0 GB, 37019566080 bytes 
64 heads, 32 sectors/track, 35304 cylinders 
Units = cylinders of 2048 * 512 = 1048576 bytes 


Device Boot Start End Blocks Id System 
/dev/sdal * 1 96 98288 83 Linux 
/dev/sda2 97 35304 36052992 5 Extended 
/dev/sda5 97 1074 1001456 82 Linux swap 
/dev/sda6 1075 2052 1001456 83 Linux 
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/dev/sda7 2053 5959 4000752 83 Linux 
/dev/sda8 5960 7913. 2000880 83 Linux 
/dev/sda9 7914 17680 10001392 83 Linux 
/dev/sdal0 17681 35304 18046960 83 Linux 


To give you an idea of the layout, here is a snippet from “mount”: 


/dev/sda6 on / type ext3 (rw) 
/dev/sdal on /boot type ext3 (rw) 
/dev/sda9 on /home type ext3 (rw) 
/dev/sda7 on /usr type ext3 (rw) 
/dev/sda8 on /var type ext3 (rw) 


Note that /dev/sda10 is specifically left out. This will be the clus- 
ter partition. If you are using a different disk layout, set aside at 
least one partition where you will be placing the shared data area. 


. Run “up2date” and make sure everything has all the necessary 


security patches. 


. Download and install the kernel source I have provided on the 


TKCluster home page, or patch your own kernel as desired if you 
are using different hardware. 

If something goes wrong, do not replace your existing kernel 
from Fedora, just configure /etc/grub.conf so that you can select 
the new kernel from the menu and boot it. Once the kernel boots 
fine and you are comfortable with it, modify /etc/grub.conf so 
that it boots this new kernel without your selecting it. Make sure 
you also grab the openMosix user-land tools from their Web site 
and install them at the same time. 


. You will have to manually adjust /etc/modules.conf if the Fedora 


install does not automatically detect both the 3c940 and the Intel 
Pro/1000. If so, you will need to insert the following two lines 
into /etc/modules.conf, and comment-out any other references to 
“ethO” and “eth!” elsewhere in the file: 


alias ethO sk98lin 
alias ethl e1000 


Note that the Intel Pro/1000 MT uses the “e 1000” driver, and that 
this is specifically set to be “eth1”. 

Also, make sure that /etc/hosts is set up properly. The proper 
address setup is important to make the cluster work right. It relies 
upon /etc/hosts pointing to the right addresses and names in order 
to use the right interface to send messages: 


27 020.4 localhost. localdomain localhost 
192.168.1.201 clusterl clusterl.tntreloading.com 
92.168.1.202 cluster2 cluster2.tntreloading.com 
192.168.1.230 cluster cluster. tntreloading.com 

0.0.0.1 clusterlp clusterlp.tntreloading.com 
0.0.0.2 cluster2p cluster2p.tntreloading.com 
Note the use of “192.168.1.230”. This is the IP alias that 


“floats” between the two cluster nodes. Whichever cluster 
node is currently the primary owns that IP address as an alias. 
When the primary goes down, the secondary grabs that address 
by using the “IPaddr” script from the heartbeat package. Install 
the “heartbeat” package at this point if you have not already 
done so. 


. If all has gone well and you have a bootable system that sees its 


disk properly as /dev/sda, talks to both NICS, and has an entry for 
/proc/hpc (the openMosix area under /proc), you are now ready to 
configure the specific features of openMosix. The Intel Pro/1000 
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card will be used as the cluster NIC, where both openMosix and 
DRBD will talk to each other. The 3c940 will be used for talking 
to the rest of the LAN. 

The rest of this document will assume that the Intel card on 
this machine is using the address 10.0.0.1 netmask 
255.255.255.0, and the 3c940 is using 192.168.1.201 netmask 
255.255.255.0. When the secondary machine is brought up, it 
will be configured similarly, with the Intel card as eth] on 
10.0.0.2/255.255.255.0 and the 3c940 as ethO on 
192.168. 1.202/255.255.255.0. The Intel cards in each machine 
will be connected directly to one another via an MDI/X 
“crossover” UTP cable. 

7. Make sure /etc/openmosix/openmosix.config has the following 
parameters turned on: 


AUTODISCIF=eth1 
MYOMID=1 
MFS=yes 


8. Create /etc/openmosix.map with the following entries, which 
tell openMosix to set up 10.0.0.1 as node #1, and 10.0.0.2 as 
node #2: 


1 10.0.0.1 1 
2 10.0.0.2 1 


9. Download and install DRBD from Linbit. It should be a very 
quick and easy install. The default configuration is for two 
DRBD devices; however, you can create up to 255 DRBD 
devices by altering drbd/drbd_main.c and changing this line: 


int minor_count=2; 


to whatever number of DRBD devices you want. 

10. Alter your iptables configuration to allow DRBD’s port 
7788/tcp through, and openMosix ports of 5000-5700/udp, 
723/tcp, 4660/tcp, and 5428/udp. If you encounter trouble with 
the firewall, please consult the openMosix FAQ to make sure you 
have the most recent information on port numbers it uses. 

11. If a reboot still results in a usable machine, you should now 
duplicate the disk onto the second disk, which will go into the 
other machine, with dd: 


dd if=/dev/sda of=/dev/sdb bs=128M 


12. When dd is finished, shut down and move /dev/sdb into the 
other machine. Boot it up, and change its IP addresses to the 
previously mentioned addresses. Also make sure to alter 
/etc/openmosix/openmosix.config and set “MYOMID=2” so that 
openMosix knows it is running on the secondary node. 

13. Now, when both machines boot, you will have an openMosix 
cluster that is capable of sharing process load between them. If 
you run “mosmon”, you should see both “1” and “2” along the 
bottom edge of the display, meaning that both nodes are up and 
visible to openMosix. 

14. If you have an identical disk layout to the one shown here, for- 
mat the /dev/sdal0 partition on the primary machine with the 
ext3 filesystem. If you are not using an identical layout, locate the 
free space you set aside on your disk during install and format it. 

Once formatted, I suggest mounting it to /opt. (Make sure /opt 
is NOT listed in /etc/fstab, as you do not want to let the OS have 
any direct control over mounting, fsck’ing, etc. that partition. 
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TKCluster must be the only thing controlling access to that 
partition.) Then, place in it some data such as a PostgreSQL data 
directory, a MySQL data directory, some Web home directories, 
or maybe sharing /opt via NFS or Samba. Configure the related 
servers or their init scripts to point to that directory structure as 
necessary. You will need to write your own init script for 
/dev/nbO, modeled after the sample “runme” in the TKCluster 
package, so that it starts and stops all relevant servers using their 
own default init scripts. 

Because TKCluster will have control over these services and 
not the OS itself, remove the symlinks in /etc/re3.d and /ete/rce5.d 
to those services so that they do not start or stop prematurely. 
TKCluster will be in charge of making the partition available to 
the services and then starting the services when ready. Make sure 
all modifications made outside of /opt are done on both 
machines. 

15. Set up TKCluster on both machines. The default configuration 
file contained in the package is “cluster.conf’ and contains a 
setup compatible with this example setup. If you use a configura- 
tion identical to or very similar to this one, the only parameter 
you will need to edit will be the “PRIMARYSERVICES” line 
that references /dev/nb0. Point this to a script that accepts “start” 
and “stop” parameters on the command line, and that accordingly 
starts and stops the services that will be accessing the data in the 
partition “/dev/nbO”. Each partition you share between the cluster 
nodes will have its own start/stop script. 

Contained in the package is the script “installme”. Once you 
have edited cluster.conf appropriately and created your start/stop 
script, just run “‘./installme” and it will copy the files to the right 
directories. If you have a version of Perl other than 5.8.3, or if 
you have installed Perl in somewhere other than /usr, you will 
need to make some slight changes to the script. 

16. Test DRBD to make sure the firewall and DRBD are working 
properly and that data replication can take place properly. On the 
primary node: 


# modprobe drbd 

# drbdsetup /dev/nb0 disk /dev/sdal0 

# drbdsetup /dev/nb0 net 10.0.0.1:7788 10.0.0.2:7788 C \ 
-s 16384 --sync-nice -19 

# drbdsetup /dev/nb0 primary 


and then on the secondary node: 


# modprobe drbd 

# drbdsetup /dev/nb0 disk /dev/sdal0 

# drbdsetup /dev/nbO net 10.0.0.2:7788 10.0.0.1:7788 C \ 
-s 16384 --sync-nice -19 


You will now see occasional blinks of the hard drive activity 
lights in unison on both machines. The default sync rate is only 
250KB/sec. To change that to a higher throughput, do this on the 
primary node: 


# drbdsetup /dev/nb0 syncer --max 600000 
# drbdsetup /dev/nb0 syncer --min 599999 


The hard drive lights on each system should now stay “on” almost 
continuously. The raw data on the primary system’s /dev/sda10 is 
flowing over the gigabit link to the secondary. To see what kind of 
rate you are getting, take a look at DRBD’s /proc entry: 
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# cat /proc/drbd 
version: 0.6.12 (api:64/proto:62) 


0: cs:SyncingAll st:Primary/Secondary ns:474328 nr:18054652 \ 
dw: 18061700 dr:482233 pe:840 ua:0 
DP esas ate nad alivan tenors J] sync'ed: 2.7% (17160/17623)M 
finish: 0:07:10h speed: 47,441 (46,201) K/sec 
1: cs:Unconfigured st:Secondary/Unknown ns:0 nr:0 dw:0 dr:0 pe:0 ua:0 


The line “finish: 0:07:10h...” tells all we need to know about the 
speed of synchronization. The first number “47,441” is the 
instantaneous rate of the current transfer in KB/sec. The number 
in parenthesis “(46,201)” is the average over the time the sync 
has been running. In this case, the sync’er setting had already 
been set to a high value before initiating the sync at 250 KB/sec, 
so the average is very close to the instantaneous. This is generally 
the case in a failover situation when the secondary takes over the 
role of primary. 

17. We have now verified that both openMosix and drbd are 
working as desired. At this point, drbd can be stopped and the 
drbd.o module unloaded on both machines, as TKCluster does 
this job as part of its startup. After that, TKCluster can be started 
on the primary machine by running its init script: 


# /etc/init.d/storagecluster start 


By foregoing food and sleep, Fred is never more than 
22.5 hours away from any of his 317 racks of servers. 
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You can watch the progress of TKCluster in /var/log/cluster.log. 
If everything is set up right and your services are starting prop- 
erly, you will see something like the following: 


.../usr/local/bin/checkinitdead.p1:3992: \ 
Starting /usr/local/bin/checkinitdead.p1 
.../usr/local/bin/checkinitdead.pl:3992: I am clusterip, \ 
clusterlp is preferred storage master 
.../usr/local/bin/checkinitdead.p1:3992: returning immediately \ 
to become master 


.../usr/local/bin/becomestorage.p1:3997: role is initially set to 
.../usr/local/bin/becomestorage.p1:3997: "2" is up 
.../usr/local/bin/storaged.p1:3990: /usr/local/bin/storaged.pl \ 


starting 


.../usr/local/bin/storaged.p1:3990: writing my wait state to \ 


/tmp/storaged.waitstatus as "waiting" 


.../usr/local/bin/storaged.p1:3990: server started on port 2345 
.../usr/local/bin/becomestorage.p1:3997: 10.0.0.2 waitstatus \ 


is nowait (returned ) 


.../usr/local/bin/becomestorage.p1:3997: "10.0.0.2" is not \ 


waiting for me, so I'll become primary 


.../usr/local/bin/storaged.pl:4011: writing my wait state to \ 


/tmp/storaged.waitstatus as “waiting” 


.../usr/local/bin/becomestorage.p1:3997: taking cluster IP: \ 


/etc/ha.d/resource.d/IPaddr 192.168.254.230 start >/dev/null 2>&1 


.../usr/local/bin/storaged.p1:4073: returning getwait() with "waiting" 
.../usr/local/bin/storaged.pl:4081: writing my wait state to \ 


/tmp/storaged.waitstatus as "waiting" 


Next time he should install a 
Remote KVM™ from Lantronix... 


Fred is at it again—traipsing all over the lower 48 just to 
reboot a couple of servers. He would be much better off 
if he simply attached each of his servers to a Lantronix 
Remote KVM. Remote KVM gives you total control over 
your servers no matter where they are. You get a secure 
keyboard-video-mouse interface from anywhere in 

the world using nothing more than a standard TCP/IP 
connection and a browser. No extra software to buy, no 
fancy new hardware. And besides, you get to sleep in 
your own bed. 


To find out more about how a Lantronix Remote KVM 
would help Fred cut down on Midnight Snack Pies, visit 
www.NoFredNo.com. 

© YOU KNOW 

EONE 


Tell us the stor 
| win an MP3 


(TRONIX’ 


15353 Barranca Parkway, Irvine, CA 92618, USA 
©2004 Lantronix, Inc. Lantronix and Remote KVM are registered trademarks of Lantronix, Inc. 
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.../usr/local/bin/becomestorage.p1:3997: EXEC: /sbin/e2fsck \ 
-p /dev/sdal0 

.../usr/local/bin/becomestorage.p1:3997: RET: /sbin/e2fsck \ 
-p /dev/sdal0 0 


Each TKCluster process reports date and time (replaced by “...” 
here), the full pathname, and its PID on each line of the logfile. In 
many cases where TKCluster is calling an external program, the 
word “EXEC: prefixes the command. When the command 
returns, the message includes “RET:”, followed by the command, 
with the exit status appended to the end (note the “0” on the last 
line of the above snippet). After this, you would see various 
“EXEC” and “RET” pairs for loading drbd, configuring the drbd 
device, and so on. 


Now that the primary machine is up, you can run the same init script 
on the secondary and watch /var/log/cluster.log on the secondary 
machine as the corresponding processes start there. 


Testing 

TKCluster is relatively easy to test. While both machines are up 
and after the initial drbd sync is done, hit the reset button on the sec- 
ondary while watching /var/log/cluster.log on the primary. You 
should see no messages appear in /var/log/cluster.log until the sec- 
ondary machine starts coming back up. When that happens, you 
will see: 


.../usr/local/bin/storaged.p1:15130: returning getwait() with "waiting" 


This says that the secondary attempted to contact the primary to 
inform it that it was coming up. The next few messages will be 
something like: 


.../usr/local/bin/monitorstorage.pl:1471: Status/Role Change \ 
from primary to primary syncing 

.../usr/local/bin/monitorstorage.pl:1471: Status/Role Change \ 
from primary syncing to primary 


TKCluster recognizes the role changes and ensures that DRBD is in 
a state where it will accept connections from the remote side for 
resynchronization. It is during this that a “fast” sync will occur. 

Once the drbd sync is done, hit the reset switch on the primary 
while watching /var/log/cluster.log on the secondary. You should 
see something like this: 


.../usr/local/bin/monitorstorage.pl:4029: MOSIX not alive on \ 
node 1 (clusterlp) 


This message will be repeated by one or more “monitorstorage.pl” 
processes until the number of re-tries configured in cluster.conf has 
been counted and “becomestorage.pl” is issued with the “grab” 
parameter, which means to forcibly attempt to reboot the primary 
node and grab all its services: 


.../usr/local/bin/monitorstorage.pl:4029: MOSIX not alive on \ 
node 1 (cluster1p) 
.../usr/local/bin/monitorstorage.pl:4014: getting out of retry loop 
.../usr/local/bin/monitorstorage.pl:4014: Falling out of \ 
monitorstorage.pl. I AM NOT PRIMARY! 
.../usr/local/bin/monitorstorage.pl:4014: \ 
EXEC: /usr/local/bin/becomestorage.pl grab 
.../usr/local/bin/becomestorage.p1:9847: \ 
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EXEC: /usr/local/bin/getstoragestatus.p] clusterlp reboot & 
.../usr/local/bin/becomestorage.p1:9847: \ 

RET: /usr/local/bin/getstoragestatus.p] clusterlp reboot & 0 
.../usr/local/bin/becomestorage.p1:9847: \ 

EXEC: /usr/local/bin/becomestorage.pl start primary 
.../usr/local/bin/becomestorage.p1:9851: role is initially set \ 

to primary 
.../usr/local/bin/becomestorage.p]:9851: taking cluster \ 

IP: /etc/ha.d/resource.d/IPaddr 192.168.1.230 start >/dev/null 2>&1 
.../usr/local/bin/storaged.p1:9916: writing my wait state to \ 

/tmp/storaged.waitstatus as "waiting" 


After this point, you will notice that the log entries look just like 
those of the primary, because now the secondary has assumed the 
role of primary. 

To explain this, TKCluster uses a “preferred storage” and “init 
dead” design where the cluster does not necessarily assume a single 
state when booted every time. When booted, the machine that is not 
set as the “preferred” machine will wait for a period of time known 
as “init dead”. If the “init dead” time passes and the machine that is 
set as “preferred” has not yet come up, the secondary will assume 
the worst and take control of the storage cluster. If the machine set 
as “preferred” starts up after this “init dead” expires, it will assume 
the role of secondary, despite having been preferred. 

If the “preferred” machine starts up before the expiration of the 
“init dead” time, it will automatically assume its role as the primary, 
and the non-preferred machine will immediately fall into secondary 
mode. This allows for “hard” failures of the primary, such as 
purposely downing it for hardware upgrades, etc., and account- 
ing for the possibility of a power failure during the time while 
the “preferred” is down. 

If the non-preferred machine were always to assume a role of 
secondary on boot, it would never become primary without being 
told to do so. So, if the primary machine goes down and stays down, 
the secondary will take over. If a power outage or other reboot of the 
remaining “live” machine occurs, no more than the “init dead” time 
period will pass before it assumes control of the storage. During the 
“init dead” period, manual intervention can force it to skip the “init 
dead” delay and go immediately into primary mode. 


Conclusion 

Affordable, small-scale, high-availability clusters can be built 
using TKCluster. Some scripting may be necessary to customize 
TKCluster to your own needs, but the cluster framework is stable 
and reliable. 


Resources 

TKCluster Home — 
http://www.SolidRockTechnologies.com/clustermanager/ 

Heartbeat — http: //www.]inux-ha.org/download/ 

openMosix — http://www. openmosix.org/ 

drbd — http://www. linbit.com/ (free registration required) 

libata kernel patches —http://www.kernel.org/pub/linux/ \ 
kernel/people/jgarzik/libata/ 


Tom Kunz holds a degree in Mechanical Engineering and lives with his 
amazing wife and children in the Pocono region of northeastern 
Pennsylvania. He has been involved in Linux and Unix systems administra- 
tion and programming since about 1995. Tom has recently opened a 
Web/nail hosting and custom software business and his Web site is 
http://www. SolidRockTechnologies.com/. He can be reached via email 
at: tkunz@SolidRockTechnologies.com. 
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Keeping Data in Sync::rsync — Part Il 


Chris Hare 


ata management is an ongoing issue that plagues many 

companies. Regardless of the size of the enterprise, organi- 

zations are constantly trying to find ways to move data 
securely between systems in an automated fashion, keep file systems 
or data files synchronized, or simply ensure that a group of systems 
has common data. 

Keeping files synchronized in the enterprise generally means 
copying the file from one system to another. The assumption most 
organizations make is the availability of high-speed, low-latency 
networks where copying a large file is performed expediently. 

This article is the second in a two- 
part series on the data synchronization 
tool rsync. The first part examined 
what rsync is, how it works, and the rs\ Ae) Cc 
interface between the client and the SN Oe 
server. This part examines the config- 
uration of the rsync server, access 
control, and security implications. 


Configuring the rsync 
Server 


Starting the rsync daemon can be 
done by either starting the rsync 
server with the -daemon option in the 
system startup scripts, or by adding a 
line to the inetd or xinetd configura- 
tion files. The rsync daemon requires 
root privileges if you want to: 


* Use chroot() to limit file system access 
* Bind to the default TCP/IP port of 873 
° Set file ownership 


Otherwise, the rsync daemon only needs permission to read and write 
the appropriate data, log, and lock files. 

If there will be a large number of connections or high volume of 
rsync traffic, it is recommended to start rsync in standalone mode, 
similar to other servers such as the Apache Web server. This 
removes the need for inetd to start the daemon for each connection. 
If the traffic volumes and number of connections is small, then typ- 
ical inetd operation is satisfactory. 

However, if the intent is to depend upon inetd to manage the 
connections, the following line must be added to /etc/services: 


rsync 873/tcp 
inetd installations use this entry: 


rsync stream tcp nowait root /usr/bin/rsync rsyncd --daemon 
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Replace “/usr/bin/rsync” with the path to where rsync is installed 
on the specific system. Because many systems today, notably 
Linux, use the eXtended Internet Daemon, xinetd, the following 
file would be installed in the /etc/xinetd.d directory to activate 
rsync: 


# default: off 
# description: The rsync server is a good addition to 
# an ftp server, as it allows crc checksumming etc. 


service rsync 


disable = no 

socket_type = stream 

wait = no 

user = root 

server = /usr/bin/rsync 
server_args = --daemon 


log_on_failure += USERID 


Note that when making changes to 
inetd.conf or adding a file to 
/etc/xinetd.d to activate the rsync 
daemon, you must restart inetd or 
xinetd as appropriate to re-read 
and activate the configuration 
changes. 

Control and operation of the 
rsync daemon is managed through 
the file rsyned.conf, typically found at /etc/rsyncd.conf. Once the 
configuration file is set up and the rsync daemon is started, any 
number of systems can connect to make backups, mirror file sys- 
tems, distribute files, etc. 

The rsyncd.conf file uses “modules” to define what a remote 
user can access. The file format is similar to the Windows INI files, 
or to the Samba configuration file. A very simple example would 
resemble: 


[ftp] 
path = /home/ftp 
comment = ftp export area 


With this simple configuration file, we can test the connectivity to 
the server: 


[chare@gw etc]$ rsync localhost:: 


ftp ftp export area 
[chare@gw etc]$ 
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The module name is “ftp” in this exam- 
ple. Using this rsyncd.conf file, a user 
would be able to connect and retrieve 
any files in the directory. It is possible 


for the user to list the available modules, 
but can a file be retrieved using this con- 
figuration? 


Parameter 


Table 1 Global Parameters 


Explanation 


motd file 


Display this file when a user connects. 


| log file 


Defines a file for messages if you don’t want to use syslog. 


syslog facility 


pid file 


Defines sthe syslog facility to use for syslog messages. 
Defines where rsync writes its process ID. 


socket options TCP socket tuning options as described on the setsocket(2) man page. 


arameter 


Table 2 Module options 
P 


Description 


Comment A text description of this module. 

path The location of the module in the file system. | 
[use chroot |Lock the user down to the directory defined in "path" just before file 

.transfer is started. 
max connections [The maximum number of connections to your rsync server. 
lock file This is the file used to support the maximum number of connections 
| option. 

read only Defines whether users can upload to the module. | 
[ list ial Defines whether users can list the module. 

uid File transfers to and from this module occur as the user. The default is 
Ip user "nobody". 

gid File that transfers to and from this module occur as this group. The 

_|default group id is "nobody". 
exclude The "exclude" option allows you to specify a space-separated list of | 


___|patterns to add to the exclude list. 


exclude from 


The "exclude from" option specifies a filename on the server that 
contains exclude patterns, one per line. 


include 


The "include" option allows you to specify a space-separated list of 
patterns that rsync should not exclude. 


= 
include/exclude. 


See also the "exclude" option above. 


| include from 


‘|The "include from" option specifies a filename on the server that 
contains include patterns, one per line. 


auth users 


The "auth users" option specifies a comma- and space-separated list 
of usernames that will be allowed to connect to this module. 


| secrets file 


The "secrets file" option specifies the name of a file that contains the 
username:password pairs used for authenticating this module. 


strict modes The "strict modes" option determines whether the permissions on the 
secrets file will be checked. 
| hosts allow The "hosts allow" option allows you to specify a list of patterns that 2 
are matched against a connecting clients hostname and IP address. 
hosts deny The "hosts deny" option allows you to specify a list of patterns that are 


matched against a connecting clients hostname and IP address. 


ignore errors 


[The “ignore errors" option tells rsyncd to ignore IO errors on the server 
when deciding whether to run the delete phase of the transfer. 


| ignore nonreadable| This tells the rsync server to completely ignore files that are not 


readable by the user. A non-readable file is a file where Unix file 
[permissions prevent the user from accessing the file. 


transfer logging 


The "transfer logging" option enables per-file logging of downloads 
_|and uploads. 


log format The "log format" option allows you to specify the format used for logging 
[file transfers when transfer logging is enabled. 
timeout The "timeout" option allows you to override the clients choice for |O 


timeout for this module. 


refuse options 


The "refuse options" option allows you to specify a space-separated list 
of rsync command-line options that will be refused by your rsync server. 


dont compress 


The "dont compress" option allows you to select filenames based on = 
wildcard patterns that should not be compressed during transfer. 
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{chare@gw chare]$ rsync localhost::ftp/ 
opendir(bin): Permission denied 


opendir(etc): Permission denied 


drwxr-xr-x 4096 2003/02/01 13:21:08 . 

U=axFeKrex 4096 2003/02/01 06:31:10 bin 
d==K="X-= 4096 2003/02/01 03:15:59 etc 
drwxr-xr-x 4096 2003/02/01 03:15:59 lib 
drwxr-sr-x 4096 2003/02/01 13:42:51 pub 


rsync error: partial transfer (code 23) at 
main.c(926) 


The example lists the files in the ftp direc- 
tory. Notice the trailing “/’, which is 
required to list the directory contents. 
Failing to include the “/” will only list the 
directory information. The partial failure 
notice occurred because some of the data 
was restricted by file permissions so it could 
not be transferred. 


[chare@gw chare]$ rsync localhost: :ftp/pub/ 


drwxr-sr-x 4096 2003/02/01 13:42:51 . 

-PW-P- ope 760 2002/12/04 23:51:50 HEADER. html 
“PWoPe Pe 187 2002/12/01 00:14:30 README. htm] 
drwxr-sr-x 4096 2002/12/01 00:14:36 family 
drwxr-sr-x 4096 2003/01/08 08:01:05 movies 
drwxr-sr-x 4096 2002/12/01 00:14:40 music 
drwxr-sr-x 4096 2003/01/14 22:58:02 slides 


[chare@gw chare]$ 


This is a similar example, listing the con- 
tents of the pub directory. 


[chare@gw chare]$ rsync 

localhost: :ftp/pub/HEADER. html 

760 2002/12/04 23:51:50 HEADER. html 
[chare@gw chare]$ rsync 

localhost: :ftp/pub/HEADER. html HEADER. html 
({chare@gw chare]$ 1s -1 HEADER* 

lchare chare 760 Mar 


“PW Poop 


=PWet--t>= 1 02:00 \ 
HEADER. htm] 


[chare@gw chare]$ 


The previous example illustrates retrieving 
the file HEADER.html. Verifying the file 
was retrieved is accomplished with the 1s -1 
command as shown, or by using a Web 
browser to display the file contents. The 
rsync server is working properly. However, 
this example doesn’t touch on the complexi- 
ties and feature set of the rsync server. 
From the small rsyncd.conf example, it is 
evident there are module parameters provid- 
ing the definition for the module. Each line 
contains a parameter name followed by an 
equals sign and the parameter value. Lines 
beginning with the traditional Unix com- 
ment character (#) are ignored. Other special 
characters for the configuration file include 
the \ character indicating that a line is con- 
tinued to the next. Otherwise, all values on 
the right side of the equals sign are assigned 
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to the parameter as either a string (no 
quotes are required) or a Boolean, which 
are assigned either a O/I 
true/false, respectively. 

As with other configuration files, there 
is a global parameters section located at the 
top of the file before the first module defin- 
ition. Defining parameters in the global part 
of the configuration file override any 
default for that parameter. The global value 
may be replaced by defining the value for a 
specific module later in the file. The global 
configuration is illustrated in this more 
complex example: 


meaning 


uid = nobody 

gid = nobody 

use chroot = no 

max connections = 4 

syslog facility = local5 

pid file = /var/run/rsyncd.pid 
motd file = /etc/rsyncd.motd 


[ftp] 
path = /var/ftp/pub 
comment = whole ftp area 
read only = yes 
list = yes 
jgnore nonreadable = yes 
{movies] 


path = /var/ftp/pub/movies 
comment = Movies and Video Clips 


The lines in the rsyncd.conf file before the 
first module form the global configuration 
section. The uid line specifies the user 
name or user id, file transfers to and from 
the given module take place when the dae- 
mon is run as root. The default is -2, which 
is the user “nobody”, and may be changed 
on a module-by-module basis. The gid line 
performs the same function, but specifies 
the group name or group id. 

The “use chroot” entry indicates 
whether the connection is limited to files 
and directory corresponding with the spe- 
cific path name. When setting it “no”, the 
user may navigate anywhere in the file sys- 
tem. The default action is “tyes”, which is 
the most secure and prevents exposing the 
system files to unauthorized users. 

The “max connections” line establishes 
how many simultaneous connections to 
rsync are allowed. The default is no limit. 
This example defines a limit of 4, mean- 
ing the fifth connection is denied with a 
message indicating the user should try 
again later. 

The “syslog facility” indicates logging 
is done using the Unix syslog service with 
the defined facility. The default facility is 
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“daemon”, although any of the available 
syslog facility levels maybe used. 

The “pid file” entry tells rsync to write 
its process ID. The “motd file” defines a 
file with a message, typically of a legal 
nature, displayed when the user connects. 

Once the global parameters are defined, 
the modules are next as shown in the 
example. 

The “path” parameter defines the loca- 
tion on the disk where the module is found. 
When a connection is made to this module, 
the user is placed in this directory. The 
“comment” parameter is what is displayed 
when the user connects and requests a listing 
of the available modules. This parameter is 
required for each module. 

The “read only” parameter indicates 
whether uploads are permitted to this mod- 
ule. If the value is “no”, then uploads are 
permitted. The “list” parameter controls 
whether the user can see this module when 
requesting a listing from the rsync server. 
And finally in this example, the “ignore 
nonreadable” parameter, when set to true, 
causes files that are not readable by the user 
to be ignored. The “ignore nonreadable” 
parameter is useful when the directory con- 
tains files the user cannot access due to file 
ownership and the associated permissions. 
Without the “ignore nonreadable” parame- 
ter, rsync will try to retrieve the file, result- 
ing in an error message the user will then 
try to resolve. 

The available global parameters are 
shown in Table 1. As well as defining 
some of these module options in the 
global configuration, the table identifies 
the parameters, which can be altered on a 
per module basis. Note that the specific 
module options (shown in Table 2) do not 
include authentication options that are dis- 
cussed later in the article. All appropriate 
module options should be applied to: 


e Provide the greatest amount of access 

control; 

limit the exposure of the rsync server, 

and; 

¢ limit the data stored within to appropriate 
or unauthorized access. 


rsync Authentication 

Rsync has a built-in authentication pro- 
tocol to provide user-based access control 
on a module-by-module basis. Typically, 
access to the rsync server is anonymous, 
meaning any user can connect and access 
the data stored in that module. Limiting 
access requires the authentication controls 
be defined to restrict global access. 
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Defining the authentication controls consists of defining a 
“secrets” file with username and clear-text password pairs, one pair 
per line. For example, the secrets file could be stored as 
/etc/rsyncd.secrets and contain: 


chare:Al8haBetsOup 
cote:4fastXfer 
hawkins : 2muchFun 


The secrets file must be stored in a protected location (ideally, 
/etc) and must be owned by root and readable only by root. 
Allowing any other user to read the password stored in this file 
eliminates any value gained from putting access control on the 
server. Once the secrets file is defined, the module must be config- 
ured to use it: 


[slides] 
path = /ftp/pub/slides 
comment = Slides and Pictures (requires authentication) 
auth users = chare, hawkins 
secrets file = /etc/rsyncd.secrets 


When the user connects to the authentication, required module, a 
“Password:” prompt is displayed and the user must provide the 
password. The default username provided is the username from the 
Shell environment, either USER or LOGNAME. The password is 
not echoed on the terminal. A sample session would resemble: 


(chare@gw chare]$ rsync Jlocalhost::slides// 


Password: 

drwxr-sr-x 4096 2003/01/14 22:58:02 . 

-PW- rw-p - 3 2003/01/15 02:20:01 .count 
-PWoPWepe- 3 2002/12/08 21:28:22 .count~ 
=Pwepeepe- 592 2002/12/01 16:47:34 HEADER. html 
Tues 187 2002/12/01 00:14:47 README. html 
-PWXP--P-- 35404504 2002/12/01 01:29:07 TRAY10.ZIP 
“PWXP> =P 30106988 2002/12/01 01:29:09 TRAY11.ZIP 
“UR I = pf = 27389104 2002/12/01 01:29:11 TRAY12.ZIP 


[chare@gw chare]$ 


The users specified in the rsynd.secrets file do not have to exist on 
the local server. This allows a remote user to have only rsync access 
to the data. Additionally, it is possible to use shell wildcards in the 
rsynd.conf file when specifying the list of authorized users. This 
simplifies managing the user list, although it can also allow access 
to modules for unintended users. 

The authentication protocol used by rsync is a 128-bit MD4 
challenge/response system. When the server receives the connec- 
tion, a challenge is created and sent to the client. When the user 
enters the password, the rsync client then generates the MD4 hash 
and transits it back to the server. The server takes the challenge 
and generates the MD4 hash using the password stored in the 
“secrets” file, and the values are compared. If they are the same, 
the password is correct and the user granted access. If the password 
is wrong, the user gets a message indicating the authentication for 
the module failed. 

The authentication transaction when sniffed over the network is 
illustrated here. In the following packet, the rsync server is telling 
the client that authentication is required for the requested module. 
The challenge is provided. The authentication required and chal- 
lenge notifications are shown in bold: 
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03:35:58.508510 localhost.rsync > localhost.33002: P 14:55(41) ack 20 
win 32767 <nop,nop,timestamp 132874500 132874499> (DF) 


0x0000 4500 005d 4ccf 4000 4006 efc9 7f00 0001 EDO. @ es ise as 
0x0010 700 0001 0369 80ea Scc0 cc40 5c5c dedc i, i..\..@\\ 
0x0020 8018 7ffF acd4e 0000 0101 080a O7eb 8104 i itstidia Nie sistance a seeseass 
0x0030 O7eb 8103 4052 5359 4e43 443a 2041 5554 «+ @RSYNCD: . AUT 
0x0040 4852 4551 4420 6462 6746 6e5a 5437 7641 HREQD.dbgFnZZ7vA 
0x0050 3454 4334 4d68 7253 4f4c 4341 0a 4TC4MhzSOLCA. 


The client passes back to the server the username (chare) and the 
MD4 hash of the password provided by the user. The username and 
MD64 hash are shown in bold: 


03:36:00.426270 localhost.33002 > localhost.rsync: P 20:49(29) ack 55 
win 32767 <nop,nop,timestamp 132875482 132874500> (DF) 


0x0000 4500 0051 5c3e 4000 4006 e066 7f00 0001 E..0\0@.@..f... 
0x0010 700 0001 80ea 0369 5c5c dede 5cc0 cc69 eee, TN Nel 
0x0020 8018 7fff 3961 0000 0101 080a O7eb 84da Beds ao Fuderesn es 
0x0030  O7eb 8104 6368 6172 6520 3636 565a 5843 os chare.66VZXC 
0x0040 5162 376f 6e59 6b63 4639 724d 366¢ 5077 Qb7onYkcN9rM6nPw 
0x0050 0a 


The rsync server responds with “OK”, indicating the authentication 
was successful and then the requested actions are performed: 


03:36:00.427694 localhost.rsync > localhost.33002: P 55:67(12) ack 49 
win 32767 <nop,nop, timestamp 132875482 132875482> (DF) 


0x0000 4500 0040 4cd0 4000 4006 efe5 7f00 0001 E..@L.@@....... 
0x0010 700 0001 0369 B0ea 5cc0 cc69 Sc5c def9 i eid TaN 
0x0020 8018 7ffFf 66c6 0000 0101 080a O7eb B4da seteaial eae gpctics eens 
0x0030 O7eb 84da 4052 5359 4e43 443a 204f 4b0a «+ @RSYNCD: .OK. 


While MD4 may be sufficient, stronger encryption may be required 
for some applications. MD4 is vulnerable to dictionary attacks [1] 
and cryptanalytic attacks against MD4 are published, so it has been 
abandoned in other applications in favor of MDS. Additionally, the 
MD64 hash is applied against only the authentication credentials — 
none of the data is encrypted. Therefore, protection from MD4 
brute force attacks and data encryption requires using rsyne with 
ssh as the shell transport. 


rsync Access Controls 

Rsync provides additional access controls using the “host allow” 
and “host deny” directives in the configuration file. The hosts allow 
option specifically grants access to the rsync server for the identi- 
fied hosts, while hosts deny specifically forbids connections. Both 
can be defined using a hostname or an IP address. If hosts allow is 
used and the connecting system does not match one of the allowed 
formats, the connection is rejected. For example, if the hosts allow 
parameter is defined as: 


[slides] 
path = /ftp/pub/slides 
comment = Slides and Pictures (requires authentication) 
auth users = chare, cote 
secrets file = /etc/rsyncd.secrets 
hosts allow = 192.168.0.3 


only connections from this IP are allowed: 


{chare@gw chare]$ rsync localhost::slides// 
@ERROR: access denied to slides from localhost (127.0.0.1) 
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rsync: connection unexpectedly closed (72 bytes read so far) 

rsync error: error in rsync protocol data stream (code 12) at jo0.c(150) 
[chare@gw chare]$ 

[chare@gw chare]$ rsync 192.168.0.3::slides// 


Password: 

drwxr-sr-Xx 4096 2003/01/14 22:58:02 . 

-PWoPWe pe 3 2003/01/15 02:20:01 «count 
-PWoPWe pe 3 2002/12/08 21:28:22 .count~ 
PWore ope 592 2002/12/01 16:47:34 HEADER. html 
“PWePe ope 187 2002/12/01 00:14:47 README. htm] 
-PWXPo oP 35404504 2002/12/01 01:29:07 TRAY10.ZIP 


For both the hosts allow and hosts deny parameters, the following 
definitions may be used to identify a system: 


¢ A dotted decimal IPv4 address of the form a.b.c.d, or an IPv6 

address of the form a:b:c::d:e:f. In this case, the incoming 

machine’s IP address must match exactly. 

An address/mask in the form address/mask, where address is the 

IP address and mask is the number of one bit in the netmask. All 

IP addresses that match the masked IP address will be allowed or 

denied depending upon the parameter. 

¢ An address/mask in the form address/mask-address, where 
address is the IP address and mask-address is the netmask in dot- 
ted decimal notation for IPv4, or similar for IPv6 (e.g., 
FEE FEFE-fEFE-FEFE:: instead of /64). All IP addresses that match the 
masked IP address will be allowed or 
denied depending upon the parameter. 

¢ A hostname. The hostname, as deter- 
mined by a reverse lookup, will be 
matched (case insensitive) against the 
pattern. Only an exact match is allowed. 

¢ A hostname pattern using wildcards. 
These are matched using the same rules 
as normal Unix filename matching. If the 
pattern matches then the client is 
allowed. 


The hosts allow and hosts deny can both be 
used in the same module definition. If both 
are defined, the hosts allow parameter is 
checked first, and a match results in the 
client being permitted to connect. If the 
host is not in hosts allow, hosts deny is 
checked and if there is a match, the client is 
denied access. If there is no match in either 
parameter, the client is allowed to connect. 

The example illustrated allows a single 
host to connect to this rsync server. 
Allowing two servers to connect requires 
using a comma-separated list of IP 
addresses. If the intent is to allow all 
machines on a given subnet access, then use 
of the IP address/netmask syntax is prefer- 
able. For example: 


hosts allow = 192.168.0.3, 192.168.0.100 
hosts allow = 192.168.0/24, \ 
192.168.10/255.255.255.0 
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Bear in mind, the default is to allow all hosts to connect to the rsync 
server. 


Making a Connection 

Aside from making a connection using rsync and the remote 
shell transport, connections to the rsync server itself can be made in 
a number of ways to suit the specific connectivity requirements. The 
connectivity options include the following: 


* Connecting to the rsync server without a remote shell transport 
* Connecting to the rsync server using a remote shell transport 
* Running an rsync server over a remote shell transport 


Connecting to the rsync server without the remote transport has 
been illustrated in a number of examples; however, it will be 
explained here again for clarity. A connection to an rsync server 
occurs when two colons are included in the lost and pathname, 
such as: 


[chare@gw chare]$ rsync 192.168.0.3::slides// 


The two colons instruct the rsync client to connect using the rsync 
protocol to the designated server on TCP port 873. If there is no 
rsync server running, the connection will be refused. If required, 
you can run rsync through a proxy, provided the proxy is defined 
using the RSYNC_PROXY environment variable and the proxy has 
been configured to allow proxying to port 873. 
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If the configuration you require must be non-interactive and a 
password is required, the environment variable RS YNC_PASSWORD 
may be used to store the password: 


{chare@alpha chare]$ RSYNC_PASSWORD="2bad4UandME" 
[chare@alpha chare]$ export RSYNC_PASSWORD 
[chare@alpha chare]$ env 

TERM=xterm 

SHELL=/bin/bash 

SSH_CLIENT=192.168.0.12 3494 22 
SSH_TTY=/dev/pts/2 

USER=chare 

MAIL=/var/spool/mail/chare 
PATH=/usr/local/bin:/bin: /usr/bin:/usr/X11R6/bin:/home/chare/bin 
PWD=/home/chare 

HOME=/home/chare 

RSYNC_PASSWORD=2bad4UandME 

LOGNAME=chare 

[chare@alpha chare]$ 


The password is only used for authenticating to the rsync daemon 
— it is not used to provide a password for the remote shell. Use cau- 
tion when using the RSYNC_PASSWORD environment variable 
because some systems allow users to see all environment variables. 
In this case, the rsyne password would be exposed. Consequently, 
use of this environment variable is not recommended. 

However, since rsync itself provides no data encryption capabil- 
ities, it is often desirable to connect to the remote rsync server and 
still have all the rsyne server functionality while using a transport 
such as ssh. In this case, the user must provide the remote shell 
transport command, the remote user, and the rsync user. The format 
for this command is: 


$ rsync -av -rsh="ssh -1 ssh-user" rsync-user@host: :module[/path]local-path 


This command line uses the rsync -rsh option to specify ssh as the 
remote transport. Alternatively, the RSYNC_RSH environment 
variable can be used to set the remote shell transport information. 
The -1 option to ssh indicates the specified user is to be used to log 
in to the remote system. 

If the remote rsync user is different from the user executing the 
command, the account for the remote rsync server must be specified 
by preceding the hostname with the username. The distinction to 
note here is the ssh-user will be used to access the system through 
the ssh transport, while the rsync-user will be used to access the 
rsync server. 

Note that if you need this functionality, you must use rsync 2.5.6 
or higher. Rsync 2.5.5 did not support this operation. 

Using the rsync server with a transport such as ssh eliminates the 
requirement for configuration of the rsync server within inetd. The 
ssh command model supports using the “command=COMMAND” 
entries in the remote user’s SSH authorized_keys entries, making it 
possible to use single-use keys to establish the connection, start the 
resync server on the remote system, and transfer the files. 

The “command” entry in the SSH authorized_keys file would 
be: 


rsync --server --daemon . 
The trailing “.” in the command entry is important, because rsync is 
expecting it for proper operation and is a requirement for the SSH 


configuration. 
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Security Implications 
As you may have noticed in this article, there are security 
implications when using rsync. These implications are: 


¢ Use of MD4 as the authentication challenge/response hash. While 
MD4 is not subject to brute force attacks (because 128-bit keys 
are still considered an “unreasonably hard problem”), there are 
cryptanalytic attacks capable of exposing the user’s password. 
However, MD4 is subject to a dictionary attack, which is often the 
method of choice by attackers when the hash or ciphertext is 
available. Although MD4 is not prone to brute-force attacks, 
many applications have replaced it with MDS. 

¢ The authentication “secrets” are stored in clear text in a file. 
Passwords should never be stored in clear text. However, rsync 
can be configured to “force” the permissions on the secrets file. If 
the permissions are incorrect, specifically meaning “others” can 
read it, the authentication fails as shown in this log file entry: 


Mar 1 04:47:45 gw rsyncd[18585]: secrets file must not be \ 
other-accessible (see strict modes option) 

Mar 1 04:47:45 gw rsyncd[18585]: continuing without secrets file 

Mar 1 04:47:45 gw rsyncd[18585]: auth failed on module slides \ 
from gw.chare-cissp.com (192.168.0.3) 


Consequently, it is likely that incorrect permissions will be cap- 

tured quickly, but not until the passwords in the secrets file have 

been captured. 
* There is no encryption of data as it is transmitted. Arguably, this 
may not be a function of rsync, although there may be some plans 
to include SSL/TLS-based session encryption in future revisions 
of rsync. At this time, however, any requirement for encryption to 
protect the data as it moves across the LAN, corporate wide area 
network, and especially the Internet, should be satisfied using 
Secure Shell (ssh). 
Firewall operation. Using rsync through a firewall requires the 
addition of firewalls establishing connectivity from inbound 
requests to the rsyne server. This can include several ports, 
depending upon how your organization will allow connections to 
the rsync server. The ports can include: 


Service Name _ Port 


Rsync server 873 TCP 
873 UDP 
Ssh 22 TCP 
Rsh 514 TCP 
Finally, the Sardonix Open Source auditing team 


(http://www. sardonix.org) has reviewed the rsync source code 
and no significant security vulnerabilities were found. Notably, 
the reviewers commented how the developers have approached 
their development with security in mind. However, like all devel- 
opment activities, there is still room for improvement, although 
no significant security exposures were identified. 


Summary of Benefits 

As discussed at the beginning of the article, there are extensive 
benefits for using rsync. It is open source and free, allowing the 
organization to include it in products or make changes as required 
for their specific applications. Rsync is small, and supports data 
transfers, even of large files, over very slow links, including analog 
modem connections. 
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Rsync can be used over secure transports, allowing for its 
use in transferring data over the Internet and can provide file 
distribution capabilities like FTP, albeit with a more compact 
protocol. Consequently, rsync is a viable tool for a wide range 
of applications including data sharing, application distribution, 
and file synchronization. 


Conclusion 

Many data transfer methods are available in the marketplace 
today, depending upon budget and complexity. Rsync provides a 
viable alternative to the complex tools available in the commercial 
marketplace, while offering incredible reliability as a data transport. 

The security risks are well understood with rsync, and when 
used with encrypted transport tools such as SSH, there is incredible 
security available to the user or organization. Additionally, the 
cross-platform availability of rsync solves some problems typically 
associated with moving data from a Windows to Unix system, or 
vice versa. 

Since rsync is an open source tool, some organizations will be 
reluctant to use it. However, there is considerable discussion in var- 
ious rsync mailing lists regarding features, bugs, enhancements, and 
general “how to” questions, for even the most avid reader to still 
learn something about rsync implementation and use. 


Resources 

There is a mailing list for the discussion of rsync and its applica- 
tions. It is open to anyone to join. See the Web page at: 
http://lists.samba.org/. 


¢@ Memory Upgrade 


The main Web site for rsync is http://rsync.samba.org/. The 
main ftp site is ftp://rsync.samba.org/pub/rsync/. This is also 
available as rsync://rsync.samba.org/rsyncftp/. 
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1A dictionary attack against the challenge/response system used by rsync 
involves capturing packets between the rsync client and server. Also, an MD4 
generator is used to create MD4 hashes of all possible dictionary words and 
combinations. The created MD4 hash values are then compared against the col- 
lected hash values to determine whether the value is correct. This is a similar 
process used by the Unix password attack program, Crack. 
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Monitoring a SAN with MRTG 


Mike Scott 


torage area networks (SANs) are relatively new to the sys 
admin’s toolbox and they bring a plethora of benefits. 
Unfortunately, they also bring complexity. SAN technology 

can potentially connect a server to hundreds or even thousands of 
storage devices via a single fibre pair. Similarly, a single host with 
multiple host bus adaptors (HBAs) can generate a huge amount of 
cross-SAN traffic, potentially causing contention on shared devices. 
The flow of traffic must be managed as the SAN grows, but 
before the traffic can be effectively managed, you must be able to 
monitor activity. Traffic is often balanced across multiple HBAs for 
performance and redundancy, and 
when considering the requirements 


The only problem was that it was clear that as the number of 
switches (and hence ports) monitored by the toolkit increased, the 
MRTG configuration file would rapidly become unmanageable. 
Furthermore, it was clear that although MRTG does a great job of 
generating all the necessary graphs and HTML to our specification, 
we needed several layers of abstraction to rapidly “drill down” into 
the data when troubleshooting. 

This data should all be easily accessible by a browser, and so it 
became clear that the visualization layer would have to sit between 
the Apache server and the raw data to help the user interpret the 
results. To solve these challenges, 
two programs were written — the 


for monitoring a SAN, it is impor- 
tant to consider that an “edge node” 
(a device on the outer periphery of 
the SAN) does not relate to a host or 
a storage array, but to an HBA. 
Thus, monitoring the SAN can be a 
headache — with multiple HBAs 
and multiple paths through the SAN 
from host to storage array. 

I recently saw a potential perfor- 
mance problem at a client site where 
multiple Solaris hosts accessed a 
single EMC Symmetrix (via a series 
of Brocade switches). It was sus- 
pected that the activities of one or 
more of the hosts were adversely 
affecting the others, including an 
important production system. A more permanent monitoring solu- 
tion was planned, but unlikely to be implemented within a month. 
Thus, this project was started, and it provided visibility of the SAN 
within a few hours of its inception. 

I chose Perl and PHP to implement this solution — Perl because of 
its ability to handle complex data structures, and PHP for its easy inte- 
gration into the Apache Web server. With hindsight, it might have been 
better to stick to one language, but this project was developed in a hurry. 


Introducing MRTG 

MRTG (Multi-Router Traffic Generator) is a data gathering and 
charting tool, written by Tobias Oetiker. As its name indicates, it was 
originally developed to monitor traditional LAN and WAN devices. 

When I googled on the MRTG Web site, it became clear that 
MRTG could do almost everything that we required. Given the cor- 
rect SNMP configuration for the switches, it can query the Fibre 
Channel statistics and graph each port on the switch. MRTG also 
allows mathematical operations on the SNMP-gathered data. This is 
a key enabler for this project, as it allows for multiple ports (poten- 
tially on different switches) to be aggregated in order to compute 
the overall throughput for any given host into the SAN. 
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configurator, and the visualizer. 

The monitoring host was to be an 
existing Sun E450, running Solaris 
9, with the Apache Web server and 
Perl (as bundled by Sun). The latest 
version of PHP and MRTG were 
then installed (along with the 
required libraries, most of which are 
usefully available pre-packaged at 
(http://www. sunfreeware. com). 

Oetiker has included some 
excellent documentation on the 
MRTG Web site, so I'll not discuss 
the details of installing MRTG here. 
Once you’ve obtained the required 
libraries, it is an easy install. 

To assist with the example pro- 
vided in this article, Figure | shows a simplified diagram of a 
fictitious SAN setup and the port assignments are as follows: 


—— 
| 
| 
| 
| 
| 
| 
| 
| 
/ 
| 
| 
| 


switchAl port assignments: 


Port 7 symmA, FAO 
Port 1 —-hostAl, HBA#O 
Port 4 = hostA2, HBA#O 


switchA2 port assignments: 
Port 7 symmA, FA#1 

Port hostAl, HBA#l 

Port 12 hostA2, HBA#l1 
Port 15 switchBl, port 15 
switchBl port assignments: 


Port 3 symmB, FA#0 

Port 1 hostBl, HBA#O 

Port 9 —-hostB2, HBA#O 

Port 15 switchAl, port 15 


switchBl port assignments: 
Port 3 symmB, FA#1 

Port hostBl, HBA#l1 
Port 9 —-hostB2, HBA#F1 
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Configuration The raison d’étre for this program is to allow a SAN configuration 


The first part of the exercise is to translate the actual switch confic- to be specified in a compact human-readable format. This ensures that 
uration into the data structure at the head of the configurator program. it can be updated easily. The configurator will then take the SAN con- 
This has already been done in Listing | for the example setup. figuration and generate suitable MRTG configuration files. 


Figure 3 The visualizer — Host view including HBA 
subgraphs 
Aggregate Throughput for hosta2 


Figure 1 Example configuration 
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The first part of the program is clearly the data structure that 
specifies the SAN. This data structure could be described as 
“switch-centric”. It is very easy to accurately populate the initial 
values, as the structure can be compared directly with the output of 
the Brocade “switchShow” command, which will display which 
ports of the Fibre Channel switch are in use (of course, the Brocade 


Figure 4 The visualizer — HBA view 
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command will show the World-Wide Numbers of the connected 
devices, which may require translation). 

The data structure could easily be hived off to a separate file, or 
even integrated with a Web front-end to make updates even easier; 
however, for the sake of simplicity, it has been integrated with the 
Perl program. The remainder of the program consists of several 


Listing 1 The configurator 
#!/usr/bin/perl -w 


# SAN To MRTG Configurator 
# Mike Scott : mike@hindsight.it 


# Version 10 
# Date : Feb 2003 
HHHHHHE 


# WARNING - THIS DATASTRUCTURE IS CASE SENSITIVE 
# Make sure that hostnames match exactly... 


HHEHHHE 
Zconfiguration= ( 
"sitea" => { 
"switchal" => { 
# PORT => [HOSTNAME, HBA NAME ] 
"01" => ["hostal", "fcaw0" dl 
"04" => ["hosta2", "fcaw0" li 
"07" => ["symma", "FA-2" J 
is 
"switcha2" => { 
"01" => ["hostal”, "fcawl" 1 
"12" => ["hosta2", "fcawl" il 
"07" => ["symma", "FA-1" alt 
"15" => ["switchbl", "“portl5" J 
i 
iS 
"siteb" => { 
"switchbl" => { 
"01" => ["hostbl", "fcaw0" i; 
"09" => ["hostb2", "fcaw0" ih 
"03" => ["symmb", MEAG ip 
"15" => ["switchal", "portl5" ] 
We 
"switchb2" => { 
"06" => ["hostbl", "fcawl" iy 
"09" => ["hostb2", "fcawl" if 
"03" => ["symmb", "FA-2" ] 
} 
i 


# Some globals... 
$MRTGDIR="/export/data/MRTG"; 
# MRTG html directory 
SHTMLDIR="$MRTGDIR/htm1"; 

# MRTG config directory 
SCFGDIR="$MRTGDIR/cfg"; 

# MRTG log directory 
$LOGDIR="$MRTGDIR/1ogs"; 


{HEE HHA HHA HHAHE 
# Routine to generate the MRTG configuration per port 
sub brocade_port ($$) { 
# These two values are the OIDs from the Brocade MIB that specify 
# swFCPortTxWords and swFCPortRxWords. They represent the number 
# of 4 byte Fibrechannel words seen passing through any given port 
# (whose port number is appended to the end of the 01D) 
my $OID_INPUT="1.3.6.1.4.1.1588.2.1.1.1.6.2.1.11."5 
my $O1D_OUTPUT="1.3.6.1.4.1.1588.2.1.1.1.6.2.1.12."; 
my ($port,$switch)=@_; 


# Physical port 0 is referred to as port 1 in the 
# SNMP information 
$port++; 


# Multiply the SNMP values (swFCPortTxWords/swFCPortRxWords) by 4 to 
# get the number of bytes 


# Also notice, the ::1 - we are using SNMPvl - I noticed that the 

# Brocade 2800 appears to support v2, but our 12000 does not. 

return "( 
${OID_INPUT}${port}\&${OID_OUTPUT}${port}:public\@${switch}:::::1 * 4)" 


Sys Admin — 29 


“foreach” loops that will first invert the 
data structure so that that it can be later 
interpreted as “host-centric”, and per-host 
aggregate formulae constructed. 

When executed, the program processes 
the configuration data structure and corre- 
lates each HBA with its respective host. It 
then produces the MRTG configuration 
files (one per group). Listing 2 shows an 
example output from the configurator tool. 


For brevity, I have included only the first 
three targets of the “sitea.cfg” file. 

Note that if a “global.inc” file exists in 
the configuration directory at the time that 
the configurator is executed, it will be 
included into the group configuration files 
via an MRTG “Include” directive. This 
allows us to have a file that can specify 
options common to all groups. An example 
of this is given in Listing 3. Also note the 


use of the AddHead directive in the 
“global.inc” file to include a CSS stylesheet 
to ensure that all pages have the same look 
as the rest of the Web site. 

Once the configuration files have been 
generated, MRTG can be started according to 
the instructions on the MRTG Web site. For 
this, I chose to run it from cron every five 
minutes (preferably by a non-privileged user). 
Listing 4 shows an example crontab extract. 


nn i —g 


Listing 1 continued 
} 


{HAHAHAHAHA 
# A quick and easy routine to check if a directory exists, if not 
# then create it. 
sub checkAndMkdir ($) { 
my ($dir)=@_; 


me (le sdesain): { 
print "Warning: Creating directory $dir\n"; 
mkdir($dir,0755) || die "Error: Could not create $dir\n"; 
} 


{HAHAHAHA 
# Main Program 


&checkAndMkdir("$CFGDIR") ; 
&checkAndMkdir("$HTMLDIR") ; 
&checkAndMkdir("$HTMLDIR/group"); 
&checkAndMkdir("$LOGDIR"); 
&checkAndMkdir("$LOGDIR/group"); 


# Loop once per group (which is a general term for "site" in our context) 
foreach $group (keys %configuration) { 


# Clear the %hosts cache 
my %hosts = (); 
print "Generating $group.cfg file...\n"; 


# take a copy of the group configuration 

# Not particularly efficient, but it makes referencing the complex 
# data structure easier... 

my %conf=%{$configuration{$group}}; 


# First we need to invert the config, so we are 

# looking at it from a host-centric view, rather than 
# the switch-centric view... 

foreach $switch (keys %conf) { 


# Loop once per defined switch port 
foreach $port (sort keys %{$conf{$switch}}) { 


# Get the hostname and HBA from the config 
($hostname, $interface)=@{$conf {$switch}->{$port}}: 


# Generate a hash of hosts in the SAN 

# each hash element is a list of switches and ports 
# that the machines’ HBAs are connected to 
push(@{$hosts{$hostname}},"$switch:$port"); 


} 


# Create the HTML and log directories, if they don't already exist 
&checkAndMkdir("$HTMLDIR/group/$group"); 
&checkAndMkdir("$HTMLDIR/group/$group/img"); 
&checkAndMkdir("$LOGDIR/group/$group"); 


# Each group has its’ own MRTG config file to be processed separately 

open(CFG,">$CFGDIR/$group.cfg") || die "ERROR: Cannot open \ 
$CFGDIR/$group.cfg\n"; 

print CFG "HtmlDir: $ TMLDIR/group/$group/\n"; 

print CFG “ImageDir: $HTMLDIR/group/$group/img\n"; 

print CFG “LogDir:  $LOGDIR/group/$group/\n" 

# If a global config file exists, then include it into the 

# generated config 
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if ( -f "$CFGDIR/global.inc" ) { 
print CFG "Include: $CFGDIR/global.inc\n"; 
} 


# Generate the MRTG config for each host in the SAN 
foreach $hostname (sort keys %hosts) { 
@host_aggregate=(); 
@host_hbalist=(); 


# Generate the config for each HBA port 
foreach $hba (@{$hosts{$hostname}}) { 
($switch, $port)=split(/:/,$hba, 2); 


# Get the HBA interface name 
(undef, $interface)=@{$conf {$switch}->{$port}}; 


# Generate the simple configuration for the HBA 

print CFG "Target[${switch}_port${port}]: \ 
",brocade_port($port,$switch)."\n"; 

print CFG “Title[${switch}_port${port}]: Throughput for \ 
$switch:$port ($hostname:$interface)\n"; 

print CFG "PageTop[${switch}_port${port}]: <H1>Throughput \ 
for $switch:$port ($hostname:$interface)</H1>\n": 


# Store the MRTG config for this HBA - this will be used 

# Once the loop is finished to generate the aggregate config 

push(@host_aggregate, &brocade_port($port, $switch)); 

push(@host_hbalist,"<a href=${switch}_port${port}.htm1> \ 
<img src=img/${switch}_port${port}-day.png height=100 \ 
width=320 alt=\"${switch}:${port} to ${hostname}: \ 
${interface}\"><br><center>${switch}:${port} to \ 
${hostname}:${interface}</center></a>"); 


} 


# Take all the HBA config cached in @host_aggregate and add them 

# together 

print CFG "Target($hostname]: ".join("+",@host_aggregate)."\n"; 

print CFG “Title[$hostname]: Aggregate Throughput for $Shostname\n"; 

print CFG "PageTop[$hostname]: <H1>Aggregate Throughput for \ 
$hostname</H1>\n"; 

print CFG "PageFoot[$hostname]: <hr><table>"; 


# A nice pretty table at the foot of the aggregate page to 
# Contain thumbnails and links to the individual HBA graphs, 
# Which we stored in @host_hbalist 
my $count=0; 
my $GRIDWIDTH=2; 
foreach (@host_hbalist) { 

if ($count%$GRIDWIDTH==0) { 

print CFG "<tr>"; 


print CFG "<td>$_</td>"; 


if ($count % $GRIDWIDTH==($GRIDWIDTH-1)) { 
print CFG "</tr>"; 


$count++; 
} 


while ($count % $GRIDWIDTH gt 0) { 
print CFG "<td>&nbsp;</td>"; 
$count++; 

} 

print CFG "</tr></table>\n"; 


} 
close(CFG); 
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Visualization operations that were causing concern for __ significant changes that could impact pro- 


By this point, you should have MRTG contention on the Symmetrix arrays) duction. Monitoring the SAN from the 
up and running. It should be generating ¢ Identification of times of significant switches via SNMP is perceived as a very 
HTML pages with PNG graphs, which is activity, allowing us to feedback to the non-intrusive method (and certainly easier 
very useful but very difficult to navigate. application owners with time-based data than deploying software agents to all SAN- 
The visualizer is a short piece of PHP code in order to reschedule I/O intensive appli- connected hosts). This solution would 
that will scan the group directories and pre- cations to a quieter time likely be approved by even the strictest 
sent thumbnails in a logical and structured change-management policies. 
manner. Graphs are presented of the aggre- Often we want to monitor critical servers. MRTG is an extremely flexible tool that 
gate graphs only — meaning that the clutter In this case, it may be difficult to justify enabled us to rapidly begin monitoring the 
of the individual HBA graphs will De ggg 
masked from the user. 

When the visualizer presents a page of Listing 2 excerpt from “sitea.cfg” 
thumbnails, it links each thumbnail back to HtmlDir: /export/data/MRTG/html /group/sitea/ 
the MRTG-generated HTML page for the mageDir: /export/data/MRTG/html /group/sitea/img 

a ne x ~ LogDir: /export/data/MRTG/logs/group/sitea/ 
host’s aggregate graphs. The user can then nclude: /export/data/MRTG/cfg/global.inc 
expand the selection simply by clicking on arget(switchal_port0l]: ( 
ic Give nnibech Fe a eee 3.6.1.4.1.1588.2.1.1.1.6.2.1.11.0281.3.6.1.4.1.1588.2.1.1.1.6.2.1.12.02:public@switchal:::::1 * 4 ) 
the thumbnail graph to obtain a better view itlelswitchal_port01]: Throughput for switchal:01 (hostal:fcaw0) 
of what is going on with that machine. ageTop[switchal_port01]: <H1>Throughput for switchal:01 (hostal:fcaw0)</H1> 
Fioure 2 ee im 1 f the ot Target[switcha2_port0lJ: ( 

igure 2 shows an example of the group 1.3.6.1.4.1.1588.2.1.1.1.6.2.1.11.02&1.3.6.1.4.1.1588.2.1.1.1.6.2.1.12.02:publ ic@switcha2:::::1 * 4 ) 
view, showing all hosts and storage itle[switcha2_port01]: Throughput for switcha2:01 (hostal:fcawl) 

i 3 ‘ Th i died i 8 a: 
devices. Note that switchb1 and switchal eatery ae 01]: <H1>Throughput for switcha2:01 (hostal:fcawl)</H1> 
are listed — these graphs represent the 3.6.1.4.1,.1588.2.1.1.1.6.2.1.11.0281.3.6.1.4.1,1588.2.1.1.1.6.2.1.12.02:publ ic@switchal:::::1 * 4 )+ \ 
Eee . ( 1.3.6.1.4.1.1588.2.1.1.1.6.2.1.11.0281.3.6.1.4.1.1588.2.1.1.1.6.2.1.12.02:public@switcha2:::::1 * 4 ) 
inter-switch links. . itle[hostal]: Aggregate Throughput for hostal 

The visualizer code should be installed ageTop(hostal]: <Hi>Aggregate Throughput for hostal</H1> 
: .¢ ne ‘% > ageFoot[hostal]: <hr><table><tr><td><a href=switchal_port0l. html><img src=img/switchal_port0l-day.png \ 

IN QUE SHTMLDIR, index.php’ » and height=100 width=320 alt="switchal:01 to hostal: fcaw0"><br><center>switchal:01 to hostal:fcaw0 \ 
the Apache Web server configured with the </center></a></td><td><a href=switcha2_port0l.html><img src=img/switcha2_port01-day.png height=100 \ 
width=320 alt="switcha2:01 to hostal:fcawl"><br><center>switcha2:01 to \ 


directive “DirectoryIndex index.php” in 
order that it is picked up automatically 
when the browser requests a directory 
index. 

When the user follows the graph thumb- 
nail from the group page, the browser is 
redirected to the MRTG-produced HTML 
page for that host’s aggregate statistics (see 
Figure 3). This shows detailed aggregate 
graphs, followed by thumbnails of each 
component HBA. Each HBA thumbnail 
can similarly be expanded by clicking on 


hostal:fcawl</center></a></td></tr></tr></table> 


the graph thumbnail to give a detailed rstand Maintain I Plan for 

report of the activities of that individual Exact y ~ % LA ontro Future Growth 

adapter. See Figure 4 for an example. Happeni 9 SarCheck fully SarCheck’s Capacity 
SarCheck translates explains each of its Planning feature helps 

Conclusion pages of sar and ps recommendations, you to plan for growth, 


output into a plain providing the before slow downs or 
English or HTML information needed problems occur. 


report, complete with to take intelligent 


Enterprise monitoring packages such as 
BMC Patrol and TeamQuest are very good 
at what they do, but occasionally we need Secommendationss informed actions. 
to look at very specific sets of data for a 
short period of time while performance 
troubleshooting. Often, the easiest way to 
achieve this is to have a package such as 
MRTG available that can quickly be 
deployed for an ad hoc request. 

In this example, MRTG allowed us to 
very quickly visualize the entire SAN. We 
identified key performance facts of which 
we were not previously aware: 


CORPORATION 


e Relative load on each individual www.sarcheck.com www.aptitune.com 
Symmetrix from each server ane 


* Correlation of SAN activity against spe- 
cific events (e.g., significant data loading 
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Listing 3 global.inc 


# Server is a quad processor £450, with two instances of MRTG running - Forks 
# set to 2 to make better use of the processors 

Forks: 2 

IconDir: /icons 

WriteExpires: Yes 

Refresh: 300 

Options[_]: noinfo, growright,printrouter, pngdate,nobanner, unknaszero 
MaxBytes(_]: 134217728 

Unscaled[_]: dwmy 


LegendI[_ inanbsp; 
Legend0[_ out&nbsp; 
Legend1[_. in&nbsp; 
Legend2[_ out&nbsp; 


YLegend[_ ' port throughput 
WithPeak[_]: dwmy 


AddHead[_]: <LINK REL="STYLESHEET" HREF=/standard.css> 

XSize[_]: 600 

YSize[_]: 300 

kilof_]: 1024 

fpocaesrianienies te aot actnre see econ a esas aS atass = Ssatee pecans ~ en aoe 


Listing 4 Sample crontab entries 


0,5,10,15,20,25,30,35,40,45,50,55 * * * * /usr/local/mrtg-2/bin/mrtg \ 
/export/data/mrtg/cfg/sitea.cfg --logging=/export/data/mrtg/log/sitea.log 

1,6,11,16,21,26,31,36,41,46,51,56 * * * * /usr/local/mrtg-2/bin/mrtg \ 
/export/data/mrtg/cfg/siteb.cfg --logging=/export/data/mrtg/log/siteb.log 
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Listing 5 ‘index.php’ - The Visualizer 


<head></head><body> 

<LINK REL="STYLESHEET" HREF=/standard.css> 
<TITLEDSAN Monitoring</title> 

<center> 
<table width=100% cellspacing=0> 

<tr valign=center><td><hl>SAN Monitoring</hl></td></tr></table></center> 


<table border=1 > 
<?php 


# SAN To MRTG Visualiser 
# Mike Scott : mike@hindsight.it 


# Version 210 
# Date : Feb 2003 
# Globals 


SHTMLDIR="/export/data/mrtg/html"; 


ALE AA AMAA teh #HHHHHHHHAHHHBHE 
heck ) 


iF 
Check input parameters (if any 
# browseBy=[d|w|m|y|none] - day,week,month,year or no thumbnails 
$browseBy=$_REQUEST[ 'browseBy']; 


echo "<center>" 
if ( $browseBy=="w") { 

$showThumbnail="week"; 

echo "<h2>thumbnails : <a href=?browseBy=d>day</a> | week | \ 

<a href=?browseBy=m>month</a> | <a href=?browseBy=y>year</a> ay 

<a href=?browseBy=none>none</a></h2>\n"; 
else if ( $browseBy=="m") { 
echo "<h2>thumbnails : <a href=?browseBy=d>day</a> | \ 
<a href=?brosweBy=w>week</a> | month | <a href=?browseBy=y>year</a> | \ 
<a href=?browseBy=none>none</a></h2>\n"; 
$showThumbnail="month"; 
else if ( $browseBy=="y") { 
echo "<h2>thumbnails : <a href=?browseBy=d>day</a> | \ 
<a href=?brosweBy=w>week</a> | <a href=?browseBy=m>month</a> FEN 
year | <a href=?browseBy=none>none</a></h2>\n"; 
$showThumbnail="year"; 
else if ( $browseBy=="none") { 
echo "<h2>thumbnails : <a href=?browseBy=d>day</a> | \ 
<a href=?brosweBy=w>week</a> | <a href=?browseBy=m>month</a> | \ 
<a href=?brosweBy=y>year</a> | none</h2>\n"; 
$showThumbnail="none"; 
else 
# Default option 
$showThumbnail="day"; 


echo "</center>"; 


# We've stored all the data in the "group" subdirectory for 
# use in a future development 
$GROUPDIR="group"; 


jE HHH HHH HHH 


if ($grouphandle = opendir("$HTMLDIR/$GROUPDIR")) { 

while (false !== ($groupname = readdir($grouphandle))) { 
if ($groupname != "." && $groupname != "".." && 

is_dir("$HTMLDIR/$GROUPDIR/$groupname”)) { 


echo "<tr valign=top><td><h1>$groupname</hl></td><td><ul>"; 
if ($hosthandle = opendir("$HTMLDIR/$GROUPDIR/$groupname/")) { 


$GRIDWIDTH=2; 
$count=0; 
echo "<table>"; 
while (false !== ($filename = readdir($hosthandle))) { 
if (preg_match("/.*(2<!_port\d\d)\.html$/",$filename)) { 


if ($count % $GRIDWIDTH == 0 ) { 
echo: “<tr>": 
} 


Shostname=substr($filename,0,strlen($filename)-5); 
echo "<td><center><a href=$GROUPDIR/$groupname/$filename>\n"; 
if ($showThumbnail!="none") { 
echo "<img src=$GROUPDIR/$groupname/ img/$hostname-$showThumbnail.png \ 
width=300 height=125>\n"; 
} 
echo "<br>$hostname</a></center></td>\n": 
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SAN environment to analyze a specific problem. It is still being 
used today at the client site and is considered a valuable tool in per- 
formance monitoring. Tobias Oetiker and Dave Rand have done a 
tremendous job in developing this package, and it certainly deserves 
sys admins’ consideration. 

The code presented in this article is an example of a rapidly 
developed application and could certainly be improved upon. It is, 
however, a good example of what can be achieved by extending an 
existing generic application with tools such as PHP and Perl to suit a 
very specific set of requirements. An archive of these scripts, includ- 
ing more detail on the install instructions than was possible to 
include in this article, is available at: http://hindsight.it/san/. 
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Listing 5 continued 


if ($count % $GRIDWIDTH == ($GRIDWIDTH-1) ) { 
echo "</tr>"; 
} 
$count+t+; 
} 
} 
while ($count % $GRIDWIDTH > 0) { 
echo "<td>&nbsp;</td>"; 


if ($count % $GRIDWIDTH == ($GRIDWIDTH-1) ) { 
echo "</tr>"; 
} 
$count++; 
i 


echo "</table>"; 
closedir($hosthandle); 


} else { 
echo "ERROR: cannot open host directory $hostname\n"; 


echo "</ul></td></tr>\n"; 
} 
} 
closedir($grouphandle); 
} else { 
echo "ERROR: cannot open group directory!"; 
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NETWORKING 


Using More Tentacles of Squid 


Ralf van Dooren 


any small and medium businesses access the Internet with 
M: relatively small Internet connection (DSL, cable, or even 

POTS/ISDN). These businesses have a box, much like you 
may have at home, that handles the traffic to and from the Internet. 
The box proxies the traffic so all employees can use the Internet. 

To take the burden off this Internet connection and to control 
what employees can access, it is wise to install a caching proxy that 
proxies the request for a Web site and caches the results on local 
storage. This way, if Bob visits the same sites as Alice, a lot of con- 
tent is already local and doesn’t have to be transferred through the 
Internet connection. 

In many cases, this Internet link 
is also used for the corporate Web 
site, and is full of dynamic content 
for your (potential) customers. In 
normal circumstances, this Web 
server probably runs fine, but what 
if your company’s Web site is 
Slashdotted? In this article, [ll 
show you how to enhance the “‘stan- 
dard” usage of a caching proxy to 
also run as a Web server accelerator 
for your corporate Web server. 

Squid is a Web proxy cache. It is 
open source and is one of the most 
used proxy servers in the world to 
proxy requests and cache results. But 
as its name implies, Squid can do 
more than that. In a previous article in Sys Admin magazine, Rajeev 
Kumar described a way to firewall your corporate Web site using 
Squid (http://www.samag.com/documents/s=9023/sam0402c/). 
I’ll use a combination of these functions: offload (and firewall) your 
corporate Web site, and cache the traffic for your employees so your 
connection will have some bandwidth left. 

Off-loading your corporate Web server can be advantageous if 
your Web server is sustaining a rather heavy load. If you use 
Squid in Web Server Acceleration mode, the static content 
(images, css-sheets, etc.) won’t have to be retrieved from the Web 
server itself, and the server can then focus its CPU cycles to the 
dynamic content. Overall, the users of your corporate Web server will 
have a much better “surfing experience”, which is vital for your Web 
appearance — a slow responding Web site won’t create as much rev- 
enue as a fast loading Web site. Furthermore, as described in Kumar’s 
Squid article, this technique can be used to firewall your Web server. 


=httpd=sccelwith=prax! 
httpd eccel_ single 


Installing Squid 

As always, there are a couple of methods to install Squid. A great 
many operating systems have developed a way to (semi-)automatically 
install your software. For example, if you use FreeBSD, installing 
Squid is as simple as: 
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(freebsd)# cd /usr/ports/www/squid 
(freebsd)# make install clean 


If you haven’t installed Squid before, you’ll get a menu with 
several choices. Adjust them to your preferences. If you need to 


make changes to this configuration, use make config in 
/ust/ports/www/squid. 
Alternatively, you can download the source from 


http://www.squid-cache.org and compile the source yourself. 
This gives you the flexibility (but also the complexity) of tuning the 
Squid installation. As most configu- 
ration options are specific for your 
system setup, I won’t go into the 
details of that. 


Configuring the Proxy 

I assume that you’ve now 
installed Squid successfully. I use 
a FreeBSD ports installation as a 
path reference for files; the loca- 
tion of files on your system may be 
different. 

To begin, the caching proxy 
must be configured. I’ll only show a 
basic configuration. You can adjust 
this to your needs. Many Web sites 
have configuration examples, so 
you can search those for inspiration. 

So, let’s say we have a LAN network, 192.168.10.0/24, and all users 
on it are allowed to surf the Web. In /usr/local/etc/squid/squid.conf, 
add/change the following lines: 


http_port 192.168.10.1:3128 
acl lan_users src 192.168.10.0 255.255.255.0 
http_access allow lan_users 


You can define the string “lan_users” yourself. Just pick something 
appropriate, so you'll remember later what it is. Port 3128 will be 
the port on which Squid listens, on address 192.168.10.1 (the LAN 
side of your box). 


Configuring the Web Accelerator 

Installing Squid in front of your Web server means that users 
will connect to Squid, as if it were the Web server. So, www.your- 
company.tld should resolve to the IP address of the Squid server. In 
this setup, your Web server can only have an internal IP address; it 
doesn’t have to be reachable from the Internet. 

Because the client’s browser is exchanging traffic with the Squid 
server, Squid has to listen to port 80. Squid also needs to know 
where to find the real content, so we must configure the internal IP 
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address of the Web server. Squid also needs to be told that it should 
act both as a proxy and a Web accelerator; this is not default mode. 
To the squid.conf, add the following lines: 


http_port 80 

httpd_accel_host <ip address of web server> 
httpd_accel_with_proxy on 
httpd_accel_single_host on 


This last line ensures minimal changes in the HTTP request. 

Additionally, you must make sure the Internet users cannot use 
your proxy setup as a stepping stone; Squid should only cache the 
corporate Web site for them and not the rest of the Internet. ACLs 
within squid.conf will accomplish this task: 


acl webserver <ip address of web server>/255.255.255.255 
http_access allow webserver 


Testing the Configuration 

To test the saved configuration, we run /usr/local/sbin/squid 
-k parse. This checks the configuration for any errors. When 
there’s no output, the configuration file is valid. 

Before Squid can be started for the first time, run 
/usr/local/sbin/squid -z, which will initiate the caching 
directories. Then start Squid with no options; Squid will start 
itself in the background. Now you can test your setup. Surfing 
from the local net should work (remember to enter the proxy IP 
and port in the client’s browser configuration). If you get “access 
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denied” messages in your browser, there might be an ACL error. 
ACLs are processed from top to bottom in squid.conf, so make 
sure your “http_access allow” statements are set before the 
“http_access deny all”. 

The next thing to test is your corporate Web site, as seen from 
the Internet. If you already changed the DNS configuration to 
reflect the new situation, you can surf to www.yourcompany.tld, but 
you can also surf to the IP address of the Squid server. Be sure that 
you are not accessing the Web site from your LAN. If everything 
goes as planned, you’ll see your corporate Web site appearing on 
your screen. 


Not from Your LAN? 

The solution presented has one major flaw — if you now try to 
access the corporate Web site from your LAN through the Squid 
server, you’ll get an “Access Denied”. If you look in your cache.log 
file, you’ll see “Forwarding loop detected” errors. This cannot be 
avoided because what actually happens is that the proxy asks itself 
for a page, which results in a forwarding loop. 

There are several options to circumvent this. If you use an internal 
DNS view, which is different from the DNS view on the Internet, 
you could change the IP address internally to the Web server, 
instead of the Squid server. Alternatively, you can exclude 
www.yourcompany.tld from the proxy list, though this will require 
manually reconfiguring each browser. Some browsers also support 
PAC scripts as a means of reconfiguring their browser settings. 
Here’s an example of such a PAC script: 


function FindProxyForURL(url, host) 

{ 

if (isInNet(host, "192.168.0.0", "255.255.0.0")) 
return "DIRECT"; 

else 
return "PROXY 192.168.10.1:3128"; 

} 


This will configure the browser to bypass the proxy (on port 3128) 
when the host resides in the local LAN segment. Of course, your 
client is still connecting to the Squid server (on port 80) which proxies 
the corporate Web server. 


Conclusion 

Using Squid as both a caching proxy for your LAN clients and 
an off-loading mechanism for your corporate Web server can help 
you save money on your Internet connection which also helps your 
Web server to survive a Slashdot effect. 
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Embedded Linux Router 


Tom Erfiavec 


desired project of mine. With the advent of many micro-Linux 

distributions, it has become an easy task that can be done by 
any Linux lover. My goal was a high-performance secure Internet 
gateway, so I built a PC/Linux device that could offer full speed on an 
Ethernet port and provide a firewalled connection to the Internet. 

It would be easy to run just one of the Linux Router Project solu- 
tions off a floppy disk. But the 1.44 or 1.68 MB floppy space that 
most LRP solutions provided was not sufficient for the router soft- 
ware configuration I wanted. One possibility was to move the software 
to a hard disk or to burn the software 
image onto a CD and run the router 
off the CD drive. But I decided to go 
for a solution with no moving 
mechanical parts. I wanted a flash- 
disk solution without any drives, 
connected devices, or even a cooling 
fan in order to provide an “install- 
and-forget” and completely silent 
operation. 

The goal of my project is 
sketched in Figure 1. The routing 
device must support NAT, DHCP, 
DNS, and packet filtering with 
stateful inspection. More than two 
network segments can be config- 
ured, thereby creating one or more 
demilitarized zones. Inbound NAT 
enables packet forwarding to virtu- 
ally provide services on the router while they are really running on a 
machine in the demilitarized segment. All this and more (using 
additional LRP packages) is possible with an LRP distribution. 


Heiece Linux to create a networking device has long been a 


Hardware 

The hardware I found suitable for my purpose was an older 
Cyrix MediaGX 233 set-top box, Allwell STB3036. It’s a slim-line, 
small footprint PC with power consumption low enough to survive 
without a power-supply fan, and a processor slow enough not to 
need a cooling fan (Figures 2 and 3). The STB3036 is equipped 
with M-Systems’ DiskOnChip (DoC) device providing 16 MB of 
solid-state disk space in a flash onboard, but it lacks both a floppy 
drive and a CD drive. The DoC module is situated in a corner of the 
motherboard, making it easy to extract and insert the module into 
the socket (Figure 4). 


Software 


My software selection was Jacques Nilo’s and Eric Wolzak’s 
LEAF/Bering, a derivative of Charles Steinkuehler’s Dachstein, 
itself a derivative of Dave Cinege’s original Linux Router Project. 
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The underlying Linux is a dwarfed Debian distribution. I chose 
Bering because it included Tom Eastep’s Shoreline Firewall, a 
handy configuration utility around iptables. Additionally, Jacques 
Nilo’s archive offers a number of fine software packages that are 
compatible with Bering. The documentation that comes with Bering 
is very well written, and it is not difficult to set up a customized 
environment. The prefixed acronym LEAF (Linux Embedded 
Appliance Firewall) suggests the purpose itself. 

The first obstacle I encountered was booting my hardware 
platform without a floppy drive from a Bering startup floppy 
disk. I didn’t have an external 
floppy drive so I used my desktop 
PC instead. I disconnected the 
floppy data cable from my desktop 
PC’s motherboard and connected it 
to the STB3036 motherboard. 
After powering desktop PC first 
and STB3036 second, it looked 
like a blood transfusion but it 
worked fine (Figure 5). STB3036 
woke up in the micro-Linux envi- 
ronment of Bering. As a candidate 
for embedding, it is clear that 
Bering does not run off the boot 
media but rather generates a RAM 
disk in the system’s memory, 
transfers all the software from the 
boot media into the RAM disk, and 
runs from there. 

Once my black box with micro-Linux was running, it was still 
far from operational as a router. Because a micro-Linux kernel 
supports very few devices straight out of the box, it is necessary to 
provide all the drivers for the specific hardware being used. The 
kernel that comes with Bering 1.2 is 2.4.20 and is modular. Jacques 
Nilo provides a broad range of various modular device drivers in 
his repository: 


http://leaf.sourceforge.net/devel/jnilo/bering/latest/modules/ 


To support my Ethernet interfaces, I used the drivers for RealTek 
and 3C905c-TX: 


mii.o, 8193to00.0 and 3c59x.0 


Looking ahead to the end of my project, I downloaded also the 
modules to support the DiskOnChip flash: 


mtdcore.o, docecc.o, doc2000.0, docprobe.o and nftl.o 
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Figure 1 The expected minimal functionality of an 
embedded Linux router 


ai ai a cable modem 
f NAT | DHCP/xxxx 
7 embedded [| 
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Figure 2 A slim line, low power, solid-state PC in an 
all-metal case provides a robust hardware platform for a 
networking device 


Figure 4 / prefer the DiskOnChip flash solution to 
USB flashes or CFs because it firmly sits in its socket on 
the motherboard 


38 — Sys Admin 


www. 


i 


Bering also contains support for PPP, PPPoE, HDLC, and ASYNC 
communications, which were of no interest to me. So, I deleted 
these modules from the distribution floppy and replaced them with 
the ones I needed. If you need more modules than can fit on the 
distribution floppy, an alternative approach would be to provide 
support for the flash first, then boot off the flash (as described later 
on) and add the remaining modules manually by copying them from 
the mounted floppy disk(s) to the RAM-disk. Then, you could 
re-run the configuration procedure and save the new configuration 
back to the flash boot image. 


Figure 3 The processor is cooled by a passive black 
heatsink without a fan 
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I activated the modules with the insmode command manually to 
test whether everything worked. It did. The docprobe module 
revealed the segments of my flash memory, and ntfl module gave 
life to the /dev/nftla device. However, the flash was still unusable 
without partitions and a file system. 


Configuring Modules 

So far, all the work has been done manually. To make Bering set 
the customized environment automatically, the system must be con- 
figured. This can easily be done with Ircfg, the Bering configuration 
menu system. Modules can be added at two points in the micro- 
Linux file system. The modules that do not need to be active at the 
beginning of the boot procedure (e.g., Ethernet drivers) are placed 
in /lib/modules using the “Packages configuration — modules” 
menu selection in Ircfg. However, the drivers that are necessary for 
reading from the boot device (e.g., flash memory drivers in my 
case) must be loaded prior to the boot procedure. These are placed 
in /boot/lib/modules using the “Packages configuration — initrd” 
menu selection. When the flash image is saved, these will be put 
into the initial RAM disk for startup purposes. 

Because Bering runs off the RAM disk, all the configuration 
changes done in RAM disk must be saved back to the target boot 
media before restarting the system. This can be done using the 
backup options in the Ircfg menu (Figure 6). 


Configuring Services 

The future services of the routing platform are defined and 
configured by adding software packages to the bare-bone routing 
platform. Many of them are already included in the distribution, 
such as: 


e [Ptables packet filtering 

¢ Pump (DHCP) 

¢ Shorewall configuration tool for iptables 
* Logging utility 

e DNS cache 


But there are some that may be added. I 
used the following additional packages: 


¢ libz.Irp 
° sftp.Irp 
¢ sshd.Irp 
¢ sshkey.Irp 


These provided remote management 
through SSH, although a simple Web-based 
monitoring is provided with the initial sys- 
tem (Figure 7). Some additional services 
you may want to consider include: 


Figure 6 Ircfg main menu 
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2) System configuration 


3) Packages configuration 
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h) Help 

Selection: Jj 
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¢ A tiny FreeSWAN package for IPsec VPN tunneling 
¢ A tiny HTTP server (a question of security) 
¢ A tiny SMTP server (again, a question of security) 


My selection of additional packages was too big to fit on the distri- 
bution floppy. I added them onto the RAM-disk file system after 
my machine was booting off the flash and then reconfigured the 
boot image. 


Figure 7 Weblet, a simp 
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Note that a .Irp package is nothing but a gzip archive. All LRP 
packages have a common structure and as long as they are compiled 
for the same kernel, they can be interchanged among various deriv- 
atives of the original LRP project. On Bering, I used some from 
Charles Steinkuehler’s Dachstein and they worked perfectly well. 

Besides the services you want to run, you must also acquire the 
packages needed for customizing the system. These will be used 
only once and, after the finished setup, they can be deleted from the 
system. These packages are: 


¢ fdisk.Irp 
¢ mkdosfs.Irp 
° syslinux.Irp 


They will provide target disk partitioning, creation of the file sys- 
tem, and setting up the boot procedure, respectively. 

The services can be configured individually through the Ircfg 
menu once they have been successfully installed in the RAM disk. 
The installation can be done manually by running the Irpkg com- 
mand or automatically by defining the service in the syslinux.cfg 
file, as described in the next section. 

I will not describe here how to configure the networking ser- 
vices, Shorewall firewall, and other individual packages. That 
process is very well described in the Bering documentation. 


Starting Services 

The system startup is provided by Peter Anvin’s syslinux. It can 
boot Linux from DOS FAT-12 filesystem, which is used in LRP. To 
load a service from the target disk, the service package must be 
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copied to the root of the filesystem and the service must be included 
in the LRP= parameter of the syslinux.cfg file. My line for loading 
packages is: 


LRP=root,etc,local,modules,iptables,pump,keyboard,shorwall, \ 
ulogd,dnscache,weblet,libz,sshkey,sshd,sftp 


After editing the syslinux.cfg file and saving the configuration 
changes, it’s time to start the embedding. Note that the procedure 
described so far is sufficient to boot the final configuration from a 
floppy. This setup will be able to see the flash disk as a device, but 
the flash will still be empty. 


Embedding 

With flash modules in /boot/lib/modules, the system reset will 
bring up all software needed for accessing the flash in DoC. The 
first tasks are to create a partition table in the master boot record of 
the flash disk and to create a file system. The tools for this are 
installed with: 


Irpkg -i fdisk.Irp 
Irpkg -i mkdosfs.1Irp 


fdisk is used on a flash disk in the same way as on any hard drive: 
fdisk /dev/nftla 


A partition table is created in the MBR of the flash disk, then a new 
primary partition is created. Its partition type must be changed from 
Linux type 83 to type 01 (FAT-12). The active flag must be set for 
this partition and the table saved to the flash disk /dev/nftla. 

Now a FAT-12 partition exists on the/dev/nftlal, but it is empty. 
A FAT-12 filesystem is created in it by running: 


mkdosfs -F 12 /dev/nftlal 


This makes the flash disk ready to use. It can be mounted to the root 
file system in the mountpoint /doc by: 


mkdir /doc 
mount -t msdos /dev/nftlal /doc 


The configuration that was created in the RAM disk and saved to 
the boot media (curently still the floppy disk) must now be trans- 
ferred onto the flash. The floppy is mounted and then the files 
are copied: 


mount -t msdos /dev/fd0ul680 /mnt 
cp /mnt/* /doc 


Effectively this transfers the contents of the floppy disk into the 
flash. The contents of the flash are almost ready, but it still needs 
some tweaking. Let’s look at a complete syslinux.cfg file: 


display syslinux. dpy 

timeout 0 

default linux initrd=initrd.Jrp init=/linuxrc rw root=/dev/ram0 \ 
boot=/dev/fd0ul680:msdos PKGPATH=/dev/fd0ul680 

LRP=root,etc, local ,modules,iptables,pump,keyboard,shorwall, \ 
ulogd,dnscache,weblet,libz,sshkey,sshd,sftp 
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The pointers that refer to the floppy drive must be replaced by 
pointers to the flash disk: 


boot=/dev/nftlal:msdos 
PKGPATH=/dev/nftlal 


Now the pointers are fine, but there is no bootstrap code in this 
partition. The syslinux package is used for that purpose. Syslinux 
version 1.62 never worked for me, so I used version 1.42, which 
worked fine. There are newer versions around but I had no need for 
them. To prepare the bootstrap code in the target partition, it must 
first be unmounted from the RAM disk: 


umount /doc 
syslinux -s /dev/nftlal 


The last command created a Idlinux.sys file in the root of the tar- 
get filesystem. It contains Step 3 of the bootstrap code, executed 
just after Step 2 of the bootstrap code in the boot record of the 
active partition in the flash, which runs after Step 1 bootstrap code 
in the MBR. 

The big moment has arrived. We do one last check: the partition 
table is OK; the file system is OK; the bootstrap code should be 
fine; the modules configuration is OK; the packages are OK. Power 
down. Power up. Change the boot device order in BIOS from floppy 
drive to flash. Reset. The screen flickers after the BIOS test and then 
comes the infinite darkness... 


Debugging the Bootstrap 
Procedure 


When trying to determine what went 
wrong, I suspected syslinux. I tried three 
different releases without success. When all 
reasonable resources had been tried, noth- 
ing was left but a HEX editor. I zeroed the 
Master Boot Record of the flash, restarted 
the setup from the beginning for the fourth 
time, and took snapshots of the MBR with 
dd after each step: 


dd if=/dev/nftla of=/mnt/<snapshot-name> \ 
count=1 


After this procedure, I inspected the 512-byte 
snapshots one by one with a HEX editor. I 
found out that fdisk didn’t produce the Step 1 
bootstrap code in the MBR after saving the 
partition table. With no bootstrap code in 
MBR at boot time, the execution was never 
transferred to the active partition’s boot 
record and that was the reason for the 
“infinite darkness” I experienced (Figure 8). 

I looked around and found a 16-year-old 
floppy disk with MS-DOS 6.2. The floppy 
still worked (I am not sure about MS-DOS). 
Microsoft did implement a non-docu- 
mented switch with their FDISK.COM to 
provide the Step 1 bootstrap code in the 
MBR by issuing the command: 


FDISK /MBR 
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However, this command does not address any specific device and, 
as one might expect, the bootstrap code is written to the MBR of the 
first physical disk. I could not address my flash in this way so I had 
to do the procedure manually. 

I made FDISK /MBR on a spare disk of another machine, but 
any other healthy disk with an MBR should be fine. Then I ripped 
the bootstrap code from the MBR off that hard drive by using: 


dd if=/dev/hda of=/mnt/mbr.bin bs=512 count=1 


I used a floppy disk as the transfer media and patched the bootstrap 
code over the first sector of my flash: 


mount -t msdos /dev/fd0 /mnt 
dd if=/mnt/mbr.bin of=/dev/nftla bs=512 count=1 


Then I continued the normal setup procedure on the flash for the 
fifth time, starting with the creation of a new partition table in the 
patched flash MBR with fdisk. The rest is a replay of the previous 
section. 

For those that don’t keep old floppies around or don’t want to dd 
MBRs of healthy (usually non-backuped) hard drives, I must add 
that, obviously, I did this the hard way. If I had thoroughly read the 
documentation of syslinux, I’d have known that Peter Anvin pro- 
vided a generic MBR, including the assembly language source code 
that contains the Step 1 bootstrap code. However, the Bering step- 
by-step instructions keep this fact as a secret, and you must know 
the bootstrap procedure pretty well to locate the problem. I did not 
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experiment with this generic MBR code since I trusted my patch- 
work was flawless. 

This time the reboot didn’t fail. In comparison to a floppy disk, 
the flash provided a lightning-fast bootstrap. 


Performance Evaluation 

I installed a commercial SOHO router and my embedded Linux 
router side-by-side to operate in the same environment. My evalua- 
tion test was feeding traffic in increments of 1 Mb/s on the internal 
side through the router to the external side. A probe measured the 
achieved throughput. The generated packets had the maximum size 
of 64 KB and were fragmented by the source machine to the largest 
acceptable framesize on Ethernet — 1518 bytes. In real life, the 
packets would be smaller and the routers would be busier with NAT 


address translation on additional smaller-sized packets. Still, the 
measurement can give us a feeling for performance comparison. 

Figure 9 shows the throughput graph for the commercial router. 
It was incrementally loaded with eight 1-Mb/s streams. In the 
beginning, the increments can be seen from the graph, but after the 
fourth increment (at 4 Mb/s), the linear response of the router is bro- 
ken, the graph starts oscillating, and the router starts dropping IP 
packets. The packet loss went as high as 86% with router saturation 
reached at just over 4 Mb/s. 

The situation is much different in Figure 10, which shows the 
response of the embedded Linux router. Note that the scales of the 
two graphs are different — the graph of the commercial router 
scales to 7 Mb/s, while the scale of the Linux router graph is almost 
10 times higher (60 Mb/s). The Linux router was incrementally 


Figure 8 The left snapshot is MBR after the creation with fdisk. The right snapshot is an operational MBR with Step 1 
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Figure 9 Throughput response of a commercial 
SOHO router (saturation at 4 Mb/s) _ 
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Figure 10 Throughput response of the embedded 
Linux router (no sign of saturation at 42 Mb/s) 
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loaded with 40 simultaneous streams of 1 Mb/s. The graph shows a 
nice linear response without any sign of saturation. At 42 Mb/s, ten 
times exceeding the threshold of the commercial router, the packet 
loss of the Linux router was 0% while the packet delay increased 
from | ms to 75 ms, which is still a quite acceptable response time. 


Conclusion 

An embedded LRP Linux device can be an appealing security 
gateway. This solid-state box with no moving parts and Linux hid- 
den in a flash is an “install-and-forget” solution with automatic log 
rotation that needs very little or no maintenance. Superior perfor- 
mance, compared to commercial devices, and an availability of var- 
ious configurable networking services make it ideal for an advanced 
SOHO user. 

The Bering distribution, a flavor of LRP, is one of the more 
advanced LRP branches. However, Bering already has a descendant 
called Lince, developed by Juan Jesus Prieto. It includes a down- 
sized Squid proxy server and Dansguardian Web content filtering 
package in the standard distribution. The sacrifice (or rather evolu- 
tion) is that Lince no longer fits on a floppy but on a CD-ROM. 

FreeSWAN IPsec VPN package is another logical choice for 
an embedded Linux router. But, it should be noted that 3-DES 
algorithm implemented in FreeS WAN drains the processor power 
significantly. Loaded with encrypted IPSec traffic, the performance 
of a system would be expected to drop by a factor of 10. 
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TOOLS 


Scanning Apache Logs with PHP 


Russell J.T. Dyer 


n my Web site, I have a few key Web pages located in a 
(ices requiring user authentication. For some docu- 

ments, though, I want to know when they are accessed and 
who accessed them. For instance, I might put a business proposal in 
a private directory and send emails to several prospects asking them 
to read my proposal. So I can learn 
whether the document was viewed 
and which prospects viewed it, a 
PHP script scans my Apache access 
log regularly and sends an email to 
my cell phone telling me if it dis- 
covers a match. In this installment 
of my series on PHP, I will describe 
this PHP script to explain a little 
PHP programming and to give you 
some ideas on how PHP might be 
used for systems administration and 
log monitoring. 


Getting Started 


This PHP script won’t be run 
through a browser, but will be exe- 
cuted by cron. The entry in crontab 
looks like this on my Linux server: 


0,15,30,45 * * * * root /sbin/ck-apache-log.php 


The opening four numbers separated by commas are the minute set- 
tings. The first asterisk that follows is a wildcard that means “every 
hour”. So on the hour, as well as 15, 30, and 45 minutes after the hour, 
the specified script will be run by the root user. The other three aster- 
isks represent days of the month, months of the year, and days of the 
week, respectively. That means this script will run every day. 

The opening code for the script ck-apache-log.php follows. 
Because this script will not generate a Web page display, we need to 
give the opening sha-bang (#!) along with the path for php (which 
may be different on your server), and the -q option to prevent PHP 
from involving the Web server: 


#!/usr/bin/php -q 

<?php 
$dir = '/var/log/httpd'; 
$log = "$dir/access_log"; 
$ck_log = "$dir/php-ck-log.txt"; 


$ck_list = "$dir/php-ck-list.txt"; 
$ck_iplist = "$dir/php-ck-iplist.txt"; 
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$from = 'root@dyerhouse.com'; 
$to = 'russell@dyerhouse.com' ; 


$access = array(); 
$ips = array(); 


The first variable $dir contains the 
path to where our logs and other 
files are to be stored. The variable 
$log provides the name of the 
Apache log that PHP will be scan- 
ning. The variable $ck_]og contains 
the name of the log in which PHP 
will record information on any user 
accesses that are found in the 
Apache log. The variable $ck_]ist 
names the data text file that will 
have a list of files for which we 
want our PHP script to search the 
Apache log. As for $ck_iplist, it 
contains a list of IP addresses with 
which we are familiar. This will be 
used to give a better display in the 
email messages that will be sent to 
us. The next pair of lines in our 
script establishes the variables that contain the email address of the 
server and the address of the person whom PHP will email. The last 
pair sets up two arrays, which will be filled with data later. 


Pages to Watch 

Setting aside the PHP script for a moment, let’s set up a text file 
that will contain a list of pages for which PHP is to search. This will 
be a simple text file that we’ll create with a text editor (like vi) in 
the /var/log directory and name php-ck-list.txt: 


Business Proposal|business-proposal .html 
Sales Plan|sales-plan.html 


It’s just a simple data text file with each record on a separate line. 
Each record only contains two fields of data separated by a vertical 
bar: the first is the name of the document and the second field is the 
file name. For simplicity, I’ve only listed two documents. Getting 
back to our PHP script, let’s look at the next section of code in 
which PHP will read this text file and retrieve its data: 


$PAGES = fopen("$ck_list", 'r') 
or die("Could not open page listing file."); 


$line = rtrim(fgets($PAGES, 4096)); 
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while(! feof ($PAGES) ) 

{ 
list($name,$file_name) = split('\|", $line, 2); 
$pages($name] = $file_name; 


$line = rtrim(fgets($PAGES, 4096)); 
} 
fclose($PAGES) ; 


In the first line of code above, we’re establishing the file handle 
$PAGES that will be used to read each line of text from the data 
file that we just discussed. We’re using the fopen() function to 
open the file specified in its first argument and in read mode, per 
the second argument given. If unsuccessful in opening the file, 
per the or component wrapped onto the next line, the script will 
die and display the error message we’ve provided. The next line 
uses the fgets() function to get the first line of data (i.e., 4k of 
data). Before storing that data temporarily in the variable $1 ine, 
we use the rtrim() function to trim off the right-most character, 
the line feed. 

PHP will now loop through each line of the data text file as 
long as we’re not at the end of the file. This is accomplished with 
the feof() function coupled with the exclamation point as a nega- 
tor. In the first line of the loop statement, we use the split() 
function to split out the Web page’s name and it’s file name, based 
on the vertical bar that separates them. The 1ist() function will 
capture those values and store them temporarily in the variables 
named. Next, PHP stores these values in an associative array for 
retrieval later in the script. We end the while statement by retriev- 
ing another line from the file before starting the process over. 
When we reach the end of the text file, we use fclose() to close 
the file and to drop the file handle. 


Familiar Hosts 

The next task is to get a list of IP addresses with which we are 
already familiar. These are the IP addresses for the people that we 
emailed asking to look at our documents. Without this list, PHP 
would only be able to provide us with the IP address of the host 
that accessed the documents. The text file php-ck-iplist.txt is 
set up like php-ck-1ist.txt. The only difference is the data con- 
tent — each record will contain a field containing the expected 
user’s name and then a vertical bar and then their server’s IP 
address. Below is the section of code that will extract those 
records for use in the script: 


$LIST = fopen("$ck_iplist", 'r’) 
or die("Could not open host listing file."); 
$line = rtrim(fgets($LIST, 4096)); 


while(!feof($LIST)) 


{ 
list($host,$ip) = split('\|', $line, 2); 
$hosts[$ip] = $host; 
$line = rtrim(fgets($LIST, 4096)); 

} 

fclose($LIST); 


This section of code works like the previous section. The only dif- 
ferences are the names of the variables and the like. We’re storing 
the results here in another associative array called $hosts. 
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Incidentally, it would be a little more of an involved script, but we 
could use the host command to look up the host information on any 
IP address with which we’re not familiar. 


Scanning Apache 

Now that PHP knows the file names for which it’s searching and 
knows who should be hitting them, we can have PHP search the 
Apache log: 


foreach($pages as $page_name => $file_name) { 


$L0G = fopen("$log", 'r') 
or die("Could not open the Apache log."); 


$line = rtrim(fgets($L0G, 4096)); 


whi le(!feof($L0G)) 
{ 
if(ereg($file_name, $line)){ 
preg_match('/(\d*.\d*.\d*.\d*) \- (\w*) 
\LO\d*)\/(\w*)\/(\d(4})/', $line, $matches) ; 


$ip_addr = $matches(1]; 
$htuser = $matches[2]; 
$day = $matches[3]; 
$month = $matches[4]; 
$year = $matches(5]; 


if(lisset($htuser)){ 
$htuser = ‘Anonymous’; 
if(lin_array("$ip_addr", $ips) && $ip_addr){ 
$access["$ip_addr"] = 
"'$name' was accessed on $month $day, 
$year by $htuser from $hosts[$ip_addr] 


$ip_addr.\n\n"; 


array_push($ips, "$ip_addr"); 


} 
} 
$line = rtrim(fgets($L0G, 4096)); 
} 
fclose($L0G); 


PHP closed out the data text file containing the list of Web pages 
earlier, but it still has that information stored in the associative array 
$pages. To retrieve that information, we’re using a foreach state- 
ment above. It will go through each data pair and extract the page 
name and the file name, placing them temporarily in the variables 
$page_name and $file_name, respectively. PHP will hang onto that 
information for use in this section of code in which it will search the 
Apache log and for the next section in which it will check its log to 
make sure that it didn’t already inform us of each user access. 

We start off this foreach statement block by opening the Apache 
log and grabbing one line of text as we did in the previous sections. 
We then start a while statement in which PHP will examine the line 
of data and if it contains information on a user accessing one of our 
documents, PHP will capture the data so as to prepare to email us at 
the end of the script. 
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The first line of the while statement block uses the ereg() func- 
tion to determine whether the file name is contained in the line of 
text retrieved. If it isn’t, it will skip the statement block contained in 
the if statement and get another line of data from the Apache log. If 
the line inspected does contain the file name, PHP will use the 
preg_match() function to pick apart the data needed for the email 
message. 

In this preg_match() function, we’re using Perl-like pattern 
matching. The second argument contains the string from which 
we’re extracting data. The first argument shows the patterns con- 
tained within two forward-slashes. We’re looking for a pattern 
like this: 


12.1.1.100 - russell - [15/June/2004 


Patterns within parenthesis are captured and placed in the array 
$matches, which is named in the third argument. To capture digits, 
we use \d, and to capture letters, \w. An asterisk indicates zero or 
more of the character type that precedes it. Everything else in the 
pattern shown above equals the actual characters that PHP should 
find. So that PHP won’t be confused by the forward slashes in the 
pattern, they are escaped with a back slash, meaning PHP actually 
should look for a forward slash in the string. 

To divide the results of preg_match() among the variables in the 
lines that follow it, we use a simple sequential array data access 
method: $matches[n]. Following the variables, PHP checks 
whether there is a value in the variable $htuser using the isset() 
function. We’ll always have a user name with a password-protected 
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directory, but we include this feature in case we want to add public 
files to our list. If no user name is given when the file was accessed, 
the variable won’t be set, so PHP will set it to Anonymous. 

Before storing the information we just retrieved in a temporary 
associative array (i.e., $access) and moving onto the next line in the 
Apache log, PHP checks whether we’ ve already stored it in $access 
(described in the next paragraph). We use the function in_array() 
to see whether it is in the array. The search parameter — or rather 
the IP address — is given as the first argument; the array ($ips), 
which contains a list of IP addresses in which PHP matched during 
the running of the script, is given as the second argument. 

If this is a first time during this script that this IP address was 
found to have accessed this document, then we’ll add a line of text 
that says the document for which PHP searched was accessed by the 
specified user from the IP address found, along with a name for the 
host if known. That text is stored in the associative array called 
$access, which will be keyed on the IP address. PHP then records 
the IP address in the array $ips using the array_push() function. It 
will use this array to make sure that it doesn’t email us twice on the 
same address in the same message. The rest of the code section 
above closes out like the previous sections. 


Keeping a Log 

Let’s move on to the next section of code in which PHP will 
record its findings in its log and get the email message ready. Keep 
in mind, though, that the last squiggly bracket of this section is from 
the foreach statement from the previous section of code. That is to 
say, PHP is still processing its search of one document. Once it has 
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recorded its findings to its log and saved the information in a string 

for mailing, it will search the log again for another document: 
$RECORD = fopen("$ck_log", 'a') 

or die("Could not open PHP log."); 


foreach ($access as $ip_add => $acc_info) { 
$parameter = "$page_name|$file_name|$ip_add"; 


$results = shell_exec("grep -cs '$parameter’ $php_log"); 


if($results == 0) { 


fputs($RECORD, "$parameter\n"); 
$message = $message . "$acc_info"; 
} 
} 
fclose($RECORD) ; 


For the file handle in this section, we are opening the file for 
appending, hence the “a” in the second argument of the 
fopen( function. To loop through each message temporarily stored 
in the associative array $access, we’ll use foreach again. To do 
this, we’ll reconstruct the pattern in which we saved the data in the 
log php-ck-10g.txt and save that pattern in the variable $pattern. 
To keep it simple, we will next execute a system command grep 
using the exec() function. We’re issuing the -c option of grep to 
count the number occurrences of the pattern in the log. With the -s 
option, we suppress any error messages 
from grep. In the next line of code, we use 
an if statement to check whether the result 
of the grep is zero, meaning there are no 
entries matching the pattern. If there are no 
entries, the first line of the block for the if 
statement contains an fputs() function, 
which will write to the log file the pattern 
that was given a few lines of code above 
this line. 

The final line of this if statement stores 
the information in the variable $acc_info 
in a new variable called simply $message. 
Actually, this line appends $acc_info to 
the end of whatever is already in the vari- 
able $message, which will be empty the 
first run through. Once PHP is finished 
looping through the list of Web pages for 
which it is to search, it closes out its log file 
and the foreach statement of earlier. 


Wrapping It Up 

The script is now ready to send us an 
email, so let’s look at the final section of 
code: 


if($message) { 
mail($to, "Web Log", $message, $from); 


We use an if statement to see whether the variable $message con- 
tains anything. If it does, PHP uses the mai] () function to mail us 
the contents. The variable containing the email address in which to 
send the message is given in the first argument of the mai1() func- 
tion. The second argument is the text that will go in the subject 
line of the message. The next argument is the message, and the 
last argument is the address from which it comes. 


Conclusion 

This script could be written a little tighter, but it gives you 
some examples of PHP in action and an example of a PHP script 
that isn’t run through the usual Web interface. The language is 
fairly straightforward so that it may be easily maintained and 
improved upon by various levels of programmers. It’s also a cool 
use of PHP — it can be pretty impressive when you’re sitting at 
a coffee house talking to a friend and you receive an email on 
your cell phone saying someone has just looked at a particular 
document of yours. 


Russell Dyer is a Perl programmer, a MySQL developer, and a Web designer 
living and working on a consulting basis in New Orleans. He is also an 
adjunct instructor at a technical college where he teaches Linux and other 
open source software. He can be reached at: russel ]@dyerhouse. com. 
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PERL ADVISOR 


Lightweight Persistent Data 


Randal L. Schwartz 


requently, you have data with a strong 
FR to live. That is, your data must 

persist between invocations of your 
program and occasionally even be shared 
between simultaneous invocations. 

At the high end of this demand, we have 
entire companies devoted to creating high- 
performance, multi-user, SQL-interfaced 
databases. These databases are usually 
accessed from Perl via the DBI package, or 
by some wrapper slightly above DBI, such as 
Class::DBI or DBIx::SQLEngine. The 
details of SQL might even be entirely hidden 
away using a higher level package like Tangram or Alzabo. 

But further down the scale, there are some new solutions pop- 
ping onto the scene, which invite further observation, as well as 
some old classic solutions. For example, since Perl version 2 we’ve 
been able to put a hash out on disk with dbmopen: 


dbmopen(%HASH, “/path/on/disk”, 0644) || die; 
$HASH{“key”} = “value”; 
dbmclose(%HASH) ; 


The effect of such code is that we now have a key/value pair stored 
in an external structured file. We can later come along and reopen 
the database as a hash again, and treat it as if it were a hash with pre- 
existing values: 


dbmopen(HASH, “/path/on/disk”, 0644) || die; 
foreach $key (sort keys %HASH) { 
print “$key => $HASH{($key}\n”; 
} 
dbmclose(%HASH) ; 


While the interface was relatively simple, I wrote quite a few pro- 
grams before Perl5 came around using this storage mechanism for 
my persistence. However, this storage suffered some limitations: the 
keys and values had to be under a given size, access to the structure 
could not handle multi-user reads and writes, and the resulting data 
files were not necessarily portable to other machines (because they 
used incompatible libraries or byte orders). 

When Perl5 came long, new problems arose. No longer were we 
limited to just arrays and hashes, but we could now have complex 
data types with arbitrary structure. Luckily, the mechanism 
“behind” the dbmopen was made available directly at the Perl code 
level, through the tie operator, described in the perltie manpage. 
This let others besides Larry Wall create “magical” hashes that 
could perform actions on every fetch and store. 
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One early use of the tie mechanism was 
the MLDBM package, which could take a com- 
plex value to be assigned for a given key, and 
serialize it to a single string value, which 
could then be stored much like before. For 
example: 


use MLDBM; 

tie my %hash, ’MLDBM’ or die; 

$hash{my_array} = [1..5]; 

$hash{my_scores} = { fred => 205, \ 
barney => 195, dino => 30 }; 


As each complex data structure was stored into the hash, it was con- 
verted into a string, using Data: : Dumper, FreezeThaw, or Storable. 
If a value were fetched, it would be converted back from a string to 
the complex data structure. However, the resulting value was no 
longer related to the tied hash. For example: 


my $scores = $hash{my_scores}; 
$scores->{fred} = 215; 


would no longer affect the stored data. Instead, we got warnings on 
the MLDBM manpage to “not do this”. Also, we still had all the limita- 
tions of a standard dbmopen-style database: size limits, multi-user 
access, and non-portability. 

One solution that I used on more than one occasion was to 
take over the serialization myself, and to use Storable’s 
retrieve and nstore operations directly. My code would look 
something like: 


use Storable qw(nstore retrieve); 

my $data = retrieve(’file’); 

... perform operations with $data ... 
nstore $data, ‘file’; 


Now my $data value could be an arbitrarily complex data structure, 
and any changes I made would be completely reflected in the 
updated file. The result was that I simply had a Perl data strucure 
that persisted. 

It appears that the author of Tie::Persistent had the same idea 
to use Storable on the entire top-level structure as well, except with 
a tie wrapper instead of explicit fetch-store phases, although I can’t 
vouch for the code. In fact, I see a number of CPAN entries that all 
seemed to find similar mechanisms, but none of them seemed to 
have found the “holy grail” of object persistence, making it as 
absolutely transparent as possible in a nice portable (and hopefully 
multi-user) manner. 
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And with a promotional paragraph like that, I just had to look. It 
looks simple enough. I merely say: 


my $hash = DBM::Deep->new(“foo.db”); 
$hash->{my_array} = [1..5]; 
$hash->{my_scores} = { fred => 205, barney => 195, dino => 30 }; 


And that’s it. In my next program: 


use DBM: :Deep; 


my $hash = DBM::Deep->new(“foo.db”); 
$hash->{my_scores}->{fred} = 215; \ 
# update score 


And finally, retrieving it all: 


use DBM: :Deep; 

my $hash = DBM: :Deep->new(“foo.db”); 

print join(“, “,@{$hash->{my_array}}), “\n"; 

for (sort keys %{$hash->{my_scores}}) { 
print “$_ => $hash->{my_scores}->{$_}\n”; 

} 


which prints: 


1, 2, 3, 4, 5 
barney => 195 
dino => 30 
fred => 215 


And, in fact, that all just plain worked. I’m 
impressed. We’ve avoided the MLDBM 
problem, because the update to the nested 
data worked. And, there’s no dependency 
on traditional DBMs here, so there’s no size 
limitation or byte ordering, or even the 
need for a C compiler to install. 

I’m told, although I haven’t tested it, 
that I can also add: 


$hash->lock; 
... do some shared things ... 
$hash->unlock; 


and thereby access shared data in multiple 
processes. 

There also seems to be some cool stuff 
around encrypting or compressing the data 
as well. This definitely bears further exam- 
ination. 

The limitations of DBM::Deep seem 
rather expected. Because this is a single 
data file, it’s being locked using flock, so 
we can’t persist data for multiple users 
across machines or reliably across NFS. 
Also, we have to clean up after ourselves 
from time to time by calling an optimize 
method — otherwise, unused space starts 
accumulating in the database. 

One other recent addition to the CPAN 
also caught my eye — OOPS. Unlike 
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DBM: :Deep, OOPS uses a DBI-style database (currently only compat- 
ible with PostgreSQL, MySQL, and SQLite) for its persistent store. 
However, like DBM: :Deep, once a connection is made, you pretty 
much do anything you want with the data structure, and it gets 
reflected into the permanent storage. The database tables are created 
on request, and managed by the module transparently. 

The basic mode of 00PS looks like: 


use OOPS; 
transaction(sub { 
OOPS->initial_setup( 
dbi_dsn => "dbi:SQLite:/tmp/oops’, 
username => undef, # no matter with SQLite 
password => undef, # ditto 
) unless -s “/tmp/oops”; 


my $hash = OOPS->new( 
dbi_dsn => ‘dbi:SQLite:/tmp/oops’, 
username => undef, # no matter with SQLite 
password => undef, # ditto 

yi; 


$hash->{my_array} = [1..5]; 
$hash->{my_scores} = { fred => 205, barney => 195, dino => 30}; 
$hash->{my_scores}->{fred} = 215; # update score 


$hash->commit; 
3 
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The wrapper of transaction forces this update to all be within a 
single transaction. We fetch the data similarly: 


use QOPS; 
transaction(sub { 

my $hash = OOPS->new( 
dbi_dsn => "dbi:SQLite:/tmp/oops’, 
username => undef, # no matter with SQLite 
password => undef, # ditto 
3 


print join(“, “,@{$hash->{my_array}}), “\n”; 
for (sort keys %{$hash->{my_scores}}) { 
print “$_ => $hash->{my_scores}->{$_}\n”; 
} 
M3 


And, in fact, this retrieved exactly the values I had expected. I’ll be 
exploring these two modules in greater depth in the future, and until 
then, enjoy! 


Randal L. Schwartz is a two-decade veteran of the software industry — skilled 
in software design, system administration, security, technical writing, and 
training. He has coauthored the “must-have” standards: Programming Perl, 
Learning Perl, Learning Perl for Win32 Systems, and Effective Perl 
Programming. He’s also a frequent contributor to the Perl newsgroups, and 
has moderated comp.lang.perl.announce since its inception. Since 19835, 
Randal has owned and operated Stonehenge Consulting Services, Inc. 
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Questions and Answers 


Amy Rich 


We have a bunch of Sun machines that we install via Jumpstart. 

When a new version of Solaris comes out, we download the 
ISO images and build a new Jumpstart image from them. Since new 
versions only come out four times a year, it isn’t too painful to do 
this by hand. Lately, though, we’ve been exploring Solaris 10 via 
Solaris Express as well as keeping up with the latest production 
releases. It’s become annoying to install each set of disk images by 
hand now that we do it so frequently. I was wondering whether 


The Echo server function MAY choose not to respond to ICMP echo 
requests addressed to IP broadcast or IP multicast addresses. 


A router SHOULD have a configuration option that, if enabled, 
causes the router to silently ignore all ICMP echo requests; if pro- 


vided, this option MUST default to allowing responses. 


Looking at tepdump output will verify whether this is the issue or 


someone had already written a script SO el not. You’ll see the HP trying to send ICMP 


accomplish this before we attempted to 
(re)invent the wheel. 


Because I regularly build Jumpstart 

servers for various clients, I came up 
with a script some time back that does 
everything but download the ISO images for 
you (see Listing 1). The script is under a 
BSD-style license, so feel free to modify 
and distribute it as described. 


We have a newly installed HP/UX 

11.00 machine on our network. After a 
few minutes of uptime, it suddenly falls off 
the network. If the machine is rebooted, it 
gets another few minutes of connectivity. 
As far as I can tell, the configuration is fine 
and there’s nothing running from cron or at that would kill the con- 
nection. I realize that this question is thin on details, but I’m really 
at a loss here. 


Without some debugging details (output from netstat, ndd, ps, 

tcpdump, etc.) it’s difficult to make an accurate diagnosis. My 
first guess would be that your HP machine is performing Dead 
Gateway Detection and cannot ping the gateway. Perhaps you’ve 
turned off ICMP for security reasons? Dead Gateway Detection 
relies on section 4.3.3.6 of RFC1918, which states: 


4.3.3.6 Echo Request/Reply 


A router MUST implement an ICMP Echo server function that 
receives Echo Requests sent to the router, and sends corresponding 
Echo Replies. A router MUST be prepared to receive, reassemble 
and echo an ICMP Echo Request datagram at least as the maximum 
of 576 and the MTUs of all the connected networks. 


Submit questions to: http://www. sysadminmag.com/quest/ 
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packets to the gateway about every three 
minutes. If it receives no response, the HP 
assumes that the route is dead and disables 
it. To work around this, use ndd to disable 
Dead Gateway Detection: 


ndd -set /dev/ip ip_ire_gw_probe 0 


To make this change persist through a reboot, 
modify /etc/rc.config.d/nddconf as follows: 


TRANSPORT_NAME[0J=ip 
NDD_NAME[L0]=ip_ire_gw_probe 
NDD_VALUE[0]=0 


This information is available from the HP 
technical knowledge base at: 


eS TEI 


http://www2.itrc.hp.com/service/cki/ \ 
docDisplay.do?docLocale=en_US&docId=200000062684449 


I have an AIX machine that has no tape device. I want to create 

a backup image for disaster recovery purposes, but I’m not 
really sure how to go about it. Can I still use mksysb or do I need to 
use something like Sysback or Storix? 


As long as you have a tape or writable CD/DVD drive on 

another machine, you can create a mksysb image on disk and 
then move it to the desired media. You can NFS mount a partition 
from a central server or transfer the mksysb file to another machine 
after the fact. The command to create the image would be: 


mksysb -e -i -X /path/to/mksysb-image-name 


If you’re keeping a number of images in a centralized location, such 
as a NIM master, I suggest including the hostname and date as part 
of the filename. Sysback or Storix would also be viable alternatives 
to mksysb. 
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Listing 1 Create jumpstart server 
#1 /bin/sh 


ERE 
dE create-js-server 
dH 


Hf Copyright (c) 2002 arr@oceanwave.com 

df All rights reserved. 

dH 

# Redistribution and use in source and binary forms, with or without 
iH modification, are permitted provided that the following conditions 
## are met: 


iH} 1, Redistributions of source code must retain the above copyright 

Hf ~~ notice, this list of conditions and the following disclaimer. 

iH} 2, Redistributions in binary form must reproduce the above copyright 

Hf — notice, this list of conditions and the following disclaimer in the 
HH =~ documentation and/or other materials provided with the distribution. 
if 3. The name of the author may not be used to endorse or promote products 
HH? derived from this software without specific prior written permission. 


dif THIS SOFTWARE IS PROVIDED BY THE AUTHOR “AS IS'' AND ANY EXPRESS OR 

#Hf IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES 
tHE OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. 
df IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, 

#Hf INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT 


#HE NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 


#Ht DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 
dHE THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 

#H# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF 
dHt THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE 


iH} 
# HHH HHH HEHEHE HHH 
PEPER EEE 


# This script creates a jumpstart image from the Solaris Software CDs 1 of 2 
# and 2 of 2 ISO images and copies over the default sysidcfg file from the 
# existing Solaris 9 sysidcfg directory if none exists for the newly installed 
# 0S version. 

# 
# This script expects the ISO images of the Solaris Software CDs 1 of 2 and 2 
# of 2 to be located in a single directory in either zipped or unzipped 

# format. 

# 
# This script requires two arguments: 

# 

# * The name of the first ISO image, e.g. $o]-9-u5-sparc-vl.zip (or 

# — /full/path/to/sol-9-u5-sparc-vl.zip, if the images are not located in 
# the current directory) 

# 


# * The full path to where the jumpstart image is to be installed, 
# e.g. /install/data/cdrom/Sun0S-5.9-sparc-12.03 


# 
HAHAH HHH HHH HH HH HHH 


cdroot=/cdrom 

s0=${cdroot}/s0 

sl=${cdroot}/s1 

1fl=/dev/lofi/1 

1f2=/dev/lofi/2 
sysidcfgdir=/install/jumpstart/sysidcfg 


if [ "x$l" =x J; then 

echo "Usage: $0 <iso-image> /path/to/install/server" 
exit 0 

fi 


if [ "x$2" = x ]; then 

echo "Usage: $0 <iso-image> /path/to/install/server" 
exit 0 

fi 


path="dirname $1° 
serverroot=$2 
osver=basename ${serverroot} | awk -F- '{print $1 "-" $2}'* 


if [ "x${path}" = x. ]; then 
echo “path not set" 

path=" /usr/bin/pwd* 

fi 
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Zipimage="basename $1 -vl.zip* 
jsoimage="basename $1* 


if [ "${isoimage}" != "${zipimage}" ]; then 

imgname="basename $1 -vl.zip® 

cd ${path} 

if [ -f "${path}/${imgname}-vl.zip" ] && [ -f ${imgname}-v2.zip ]; then 
echo "Using zipped ISO images, unzipping ${imgname}-vl.zip... \c" 
unzip ${imgname}-vl.zip 1> /dev/null 
echo " ${imgname}-v2.zip” 
unzip ${imgname}-v2.zip 1> /dev/null 
echo 

else 
echo "Can not find both ${path}/${imgname} zip files, aborting.” 
exit 1 

fi 


imgname="basename $1 -vl.iso” 
fi 


if [ "${imgname}" = "${isoimage}" ]; then 

echo 

echo "You specified ${isoimage}, an invalid ISO image name.” 

echo "Be sure to use the first disk image. It should end in \"-vl.iso\" \ 
Om EVL. Zip.” 

echo 

exit 0 

fi 


if ( ! -f "${path}/${imgname}-vl.iso" ]; then 

echo 

echo "The file ${path}/${imgname}-vl.iso does not exist." 
echo "Please choose a valid ISO image and try again." 
echo 

exit 0 
elif [ ! -f "${path}/${imgname}-v2.iso" ]; then 

echo "The file ${path}/${imgname}-v2.iso does not exist, aborting.” 
echo 

exit 1 

fi 


echo "Creating sl of the first ISO image." 
echo 


if [ ! -f ${path}/${imgname}-vl-sl.iso ]; then 
string="/usr/bin/dd if=${path}/${imgname}-vl.iso bs=512 count=1 2>/dev/null| \ 
od -D -j 452 -N 8° 


startcyl=echo ${string}| awk '{print $2}"* 

sblock="/usr/bin/expr ${startcyl} \* 640° # 640 == num blocks in cyl 
size=-echo ${string}| awk '{print $3}'~ 

size="/usr/bin/expr ${size} \+ 0° 


/usr/bin/dd if=${path}/${imgname}-vl.iso of=${path}/${imgname}-vl-sl.iso \ 
bs=512 skip=${sblock} count=${size} 2>/dev/null 


echo "Slice 1 image created." 
else 

echo 

echo "${path}/${imgname}-vl-sl.iso” 
echo "already exists, using that." 
fi 


echo "Mounting both images using lofiadm." 

echo 

/usr/sbin/lofiadm -a ${path}/${imgname}-vl.iso ${1f1} 
/usr/sbin/lofiadm -a ${path}/${imgname}-v1-sl.iso ${1f2} 


/usr/bin/mkdir -p ${s0} ${s1} 


/sbin/mount -F hsfs -o ro ${1f1} ${s0} 
/sbin/mount -F ufs -o ro ${1f2} ${s1} 


echo "ISO images mounted, installing the server in:" 

echo "${serverroot}" 

echo 

if [ -e ${serverroot} ]; then 
echo "${serverroot} exists, moving to ${serverroot}.old" 
my ${serverroot} ${serverroot}.old 

fi 

/usr/bin/mkdir -p ${serverroot} 


cd ${s0} 
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Q' just upgraded all of my ports using: 


portupgrade -a -b 


When I check the port versions of everything that’s now installed 
using: 


portversion -v 


it claims that the version I have installed is newer than the port ver- 
sion. This makes no sense, because I can’t have upgraded to a version 
that’s newer than what’s available via the ports collection. I’m 
wondering whether I corrupted something when I did the upgrade. 


After you do a cvsup to update the ports collection, be sure 
to run: 


portsdb -Uu 


The portupgrade utility relies on the file /usr/ports/INDEX being up 
to date. Since this file is only rebuilt on the cvsup servers every 
month or so, the version numbers of individual ports can appear out 
of date. Running portsdb -ul rebuilds the INDEX so that it con- 
tains the current information culled from the actual ports. The same 
effect can be achieved by doing: 


cd /usr/ports 
make index 


_omp 


ae ee __ ee 
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Amy Rich, president of the Boston-based Oceanwave Consulting, Inc. 
(http://www. oceanwave.com), has been a UNIX systems administrator for 
more than 10 years. She received a BSCS at Worcester Polytechnic Institute, 
and can be reached at: qna@oceanwave. com. 


ea ae 
Listing 1 continued 
version="/bin/1s -1|grep Solaris” 
cd ${version}/Tools 
./setup_install_server ${serverroot} 
ed / 


echo 

echo "Installation of the first CDROM complete." 

echo "Unmounting the ISO images of the first CDROM and mounting the second." 
echo 


/sbin/umount ${s0} 
/sbin/umount ${s1} 
/usr/sbin/lofiadm -d ${1f1} 
/usr/sbin/lofiadm -d ${1f2} 


if [ | -f ${path}/${imgname}-v2.iso ]; then 

echo "${path}/${imgname}-v2.iso does not exist, aborting!" 
exit 0 
fi 


/usr/sbin/lofiadm -a ${path}/${imgname}-v2.iso ${1f1} 
/sbin/mount -F hsfs -o ro ${1f1} ${s0} 
cd ${s0}/${version}/Tools 


./add_to_install_server ${serverroot} 
cdi): 


echo "Installation of the second CDROM complete.” 
echo "Unmounting the ISO image of the second CDROM." 
echo 


/sbin/umount ${s0} 
/usr/sbin/lofiadm -d ${1f1} 


if [ | -f ${sysidcfgdir}/${osver}/sysidcfg ]; then 
echo "Copying over the sysidcfg file from" 

echo "${sysidcfgdir}/Sun0S-5.9 to” 

echo "${sysidcfgdir}/${osver}/sysidcfg.” 

echo 


/usr/bin/mkdir -p ${sysidcfgdir}/${osver} 
/usr/bin/cp ${sysidcfgdir}/Sun0S-5.9/sysidcfg ${sysidcfgdir}/${osver}/ 
else 
echo "${sysidcfgdir}/${osver}/sysidcfg already exists, not overwriting.” 
echo 
fi 


echo "Jumpstart disk image installation complete” 


if [ "x${zipimage}" != x ]; then 
echo "Cleaning up unzipped ISO images." 
rm ${path}/${imgname}-vl.iso ${path}/${imgname}-v1-sl.iso \ 
${path}/${imgname}-v2.iso 
fi 
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There’s the slow way and there’s 
the BlueCat Networks way. 


The Adonis DNS Management Server™ 
finds errors fast. 


With its data check, Adonis validates your data 
before you go live. 


You've only got one life, not nine. Do you 
want to waste it searching for errors? 


Let Adonis catch them for you. 
Schedule a free online demo of 


the Adonis DNS Management 
Server today. 


Visit www.bluecatnetworks.com 
or call 1-866-895-6931. 
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July 2004 www. mag.com Sys Admin — 57 


Wholesale Prices - Worldclass Performance! 


Li j\ ee Misa Pricer 


Dedicated Servers 
10Mbps Included! 


Re on Enlightened Hosting Services 


AMD Opteron / SuSE Linux 
64-bit Power. Delivered Today! 
with 10Mbps included $1 99/mol 


; @ 
"What is your next wish?" ~~~ 


1.888.221.5902 


re F 
We Manage, You Profit! 


www.catri.net/sa 


Brought to you by 
Sandy Beach 
Technologies 


Want to get rid of SPAM with no hassle? 
Visit us at www.spamrec.com and find out 
how to try us out for 30 days for FREE! 


Here are some reasons to try us out 


e 98% efficiency in catching SPAM 

e 30 day FREE trial 

e No software/hardware to install 

° Compatible with all e-mail systems 
[own domain name required] 

e Virus protection 

° We update our system for you 
regularly 

e No complex rules for users or 
administrators 

e Whitelist or Blacklist anyone 

° All stopped e-mails are accessible 

) Spam stops the moment you are 


activated with our system 


www.spamrec.com 
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LOW PRICES? 


Visit Us NOW! 
Why WAIT? 


wate Computer Systems” 


www.imonarchcomputer.com 


Sale & Clearance items 
with prices over 50% 
below cost! 


Our Customers have rated 
us for Best Prices 
& Service @ 
ResellerRatings.com 


The Answers to your Sasi ieee tes ik 
UNIX and Linux 

Systems 

Administration 

Questions, 

all in one place! 


INIX and Linux. 
ems administrators 


Get the only magazine devoted 100% to UNIX systems 
administration — solid, technical information full of ways to 
improve the performance and extend the capabilities of your 
system. Regular columns and departments also give you a 
solid look at new product releases and upgrades, career 
opportunities, and technical meetings and conferences. 
Coverage spans a variety of platforms including Linux, 
Solaris, AIX, BSD, HP-UX, and others. If you administer a 
UNIX system — Sys Admin can save you and your 
organization time and money. 


Where can you get more information? 
Visit our Web site at: 


v.sySadminmag.com/sub/ 


Discount Keycode: 2spd 
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TRAINING 
INSTITUTE 


ation Secur 


TRAINING & EDUCATION 


_ August 10-12, 2004 San Francisco, CA 


Optional Workshops /August 8, 9, 12 & 13 


A = 
Dont Miss Out... Vendor Expo/August 10 & 11 
O. Bie a aioad Trescbe > Networking opportunities Hyatt Regency San Francisco 


Featuring Security Experts from: 
Veuzen ENG Merrill Lynch Investment Register 
Vianagen Today! 
www.misti.com/websec 


Call: 508.879.7999 
E-mail: mis@misti.com 


: Sys ; Association Sponsor | Education Sponsor 
com rILMIS mm) Admin. CYBERDEFENSE | PA 


: FILMS C 
Info/Securlt TE Note 
Teale Videos Aa ri BANK =} 


Training Infosecurity and Audit Professionals for Over 25 Years 
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Ea NuxWoRLD. 


te dd CONFERENCE & EXPO 


CONFERENCE: August 2-5, 2004 EXPO: August 3 - 5, 2004 
MOSCONE CENTER ¢ SAN FRANCISCO, CA 


Where 


Meet 


Attend LinuxWorld and... 
¢ Realize Linux as more than an operating system, but as a world of applications 
° Evaluate the latest applications and innovations from leading open source companies 


* Explore interoperability issues and opportunities in open source and 
proprietary environments 


¢ Stay on the cusp of emerging technologies and the acceleration of open source adoption 
in enterprise computing 


* Review the latest open source initiatives, their deployment and successes to help make 
informed decisions for your company 


Register Online With Priority Code: D1601 


CORNERSTONE SPONSOR PLATINUM SPONSORS 


D@LL 
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WORLD EXPO 


don't miss the 
HP TECH EVENT 


of the year! 


The HP World 2004 Solutions and 
Technology Conference & Expo offers a 
uniquely powerful line-up of real-world 
tested IT solutions presented by recognized 
industry leaders and HP experts. 


TRAINING & EDUCATION 


You'll return from the conference with useable skills 
and technology updates that will add instant value to 
your IT infrastructure. Plus you'll get a roadmap from 
the experts that will prepare you for the future and 
ensure your longevity. 


The HP World 2004 Solutions and Technology 
Conference & Expo is the only HP technical training 
event you'll need this year. 


You'll have an opportunity to attend sessions on topics 
that are important to you, including: 


Adaptive Enterprise » Alpha + Enterprise Management (OpenView, Utility and Grid Computing) - 
| High Availability/Disaster Recovery + HP-UX + IA-32/x86 Servers - IT Directors/IS Managers - 
| Itanium + Linux + Middleware - MPE - Networking (Mobile Computing, Wireless, Cisco and ProCurve) - 
| NonStop + OpenVMS » Security - Storage - Tru64 UNIX > Windows 
| 


>» The only event io deliver the focused technical content, 
real-world solutions and usable skills that have made HP World 
the largest HP users conference in the Americas. 


) . The only conference produced by the leading independent 

HP WORLD "2004 © associations of HP professionals and endorsed by HP. 

Sout and Technology Conference & Expo > The only opportunity to see first-hand the latest products, 
applications and solutions on HP World's action packed expo floor. 


> The only place to rub shoulders with the industry leaders 
and HP experts that attend HP World every year. 


august 16 — 20 world.com ¢ mccormick place * chicago 
Co-Produced by: 


CLUSIVE! As a reader of Sys Admin, register today - = =35 interex a encompass 
et save an sacitonel $50! Use a Code SYS50* shared know edge» shared power a sewn user anon 


» RECOMMENDED TRAINING VENUE FOR THE 
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A five-day tutorial and refereed technical program for security 
professionals, system and network administrators, and researchers 


13? USENIX SECURITY SYMPOSIUM cot 


August 9-13, 2004" ° EN 
San Diego, CA USA » 


EDUCATION 


i . Richard Bejtlich on Network Security Monitoring 
Back to the Future” by with Open Source Tools 


William “Earl” Boebert, Brad Johnson on Network Security Profiles 


: : Moti Yung on Malicious Cryptography 
Sandia National Lab Radia Perlman on Network Security Protocols 


22 refereed papers and 2 panels on the best new 
research including Attack Containment, Software 
Protection, the Human Interface, and more... 


¢ Steve Bellovin — Nuclear Weapons, Permissive 
Action Links, and the History of Public Key 
Cryptography 

© Cindy Cohn — | Voted? How the Law Increasingly 
Restricts Independent Security Research 

e David Evans — What Biology Can (and Can't) Teach 
Us About Security 
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TRAINING & EDUCATION 


F ight fire with fir €. Learn the threats of tomorrow, today. 


Be challenged by the experts who are doing innovative work. 

Meet and network with-thousands of your peers from all'corners of 
the world at the Black Hat Briefings USA 2004— the only technical 
security event to offer you the best of all worlds. 


@ Black-Hat 


diamond platinum 


Cisco Systems Pa = Intellitactics” 
ay a 4 5 * 
BIND VIEW Quatys Qnetih Ree. Oc sntigites gy Sana Security. Foundstone 
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@stake, Inc. Launches LC 5 

Digital security company @stake, Inc. announced the latest 
release of its LOphtCrack automated password auditing and recovery 
application, LC 5. According to the company, the new version is a 
cross-platform Windows and Unix solution, and features the first 
commercially available pre-computed password tables, which can 
reduce password auditing from hours to minutes. 

LC 5 features a wizard-based interface to configure, schedule 
and run comprehensive audits on Windows (2003, XP, 2000 and 
NT versions) and Unix (multiple versions) accounts across the 
enterprise automatically and unattended. @stake’s LC 5 is 
available in a number of editions including: Professional 
Edition, which supports both Windows and Unix environments; 
Administrator Edition, which offers scheduled audits and pre- 
computed password hashes in multiple domains to accelerate 
auditing; Site Edition, for site-wide installations; and a 
Consultant Edition for multi-client assessments. 

For more information, visit: http://vww.atstake.com/c 


IOGEAR Launches Wi-Fi 802.119 
Notebook Network Card 


IOGEAR announced a new Wireless-G Notebook Network 
Card. According to the company, the 802.11g card’s compact 
antenna board design appears to be almost invisible to laptop users 
when plugged into a PC’s CardBus slot. It is now possible to leave 
the Wi-Fi card inside a notebook when traveling. The Card pro- 
vides access for Internet surfing, networking, file sharing, and 
printing in any home or office equipped with 802.11g access. 

The device comes with built-in, advanced security features for 
safe networking. It is priced at $49.95 and is available immediately. 
IOGEAR’s Wireless-G Notebook Network Card is PC-compatible 
and operates on Microsoft Windows 98/ME/2000/XP systems. It 
comes with IOGEAR’s standard three-year warranty. 

For more information, visit: http:/vww.iogear.com 


Tethernet Announced TetherFilter 

Tethernet announced the TetherFilter 100 anti-spam and anti- 
virus appliance. According to the company, TetherFilter employs 
an email message-processing engine capable of intercepting 
spam and virus messages that violate protocol and formatting 
standards. The TetherFilter appliance is compatible with all 
existing Internet email servers. 

The TetherFilter 100 is available immediately for $1949, 
offering spam and virus protection for up to 250 users with no 
per-user licensing fees. The product supports a message flow 
rate of up to 90,000 messages per hour. Support plan pricing 
begins at $249 per year. 

For more information, visit: http://vww.tethernet.com 
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Dynamic Network Factory Introduces 
TaskMaster 16000 RAID 3U Rackmount 


According to the company, TaskMaster 16000 RAID series are 
intelligent and innovative storage subsystems for mission-critical 
and high-bandwidth applications. Features include a backplane 
design for 16 hot-swappable disk drives, N+1 load-sharing power 
supplies with PFC, pure aluminum case and components, and spe- 
cially designed airflow passages. 

TaskMaster 16000 RAID is equipped with an Ultral60 SCSI 
RAID controller to offer fault-tolerant data protection. It is 100% 
transparent to the host operating system and requires no custom 
drivers. According to the company, it is a full-featured data 
protection subsystem supporting RAID levels 0, 1, 0+1, 3, 5, 
and JBOD. 

For more information, visit: http:/Avww.storageexpert.com/ 


OSM Unveils Identity Management 
Solution for Linux 


Open Systems Management Ltd. announced the general 
availability of a version for Linux environments of its COSuser 
user provisioning software. According to the company, the 
release means that mid- to large-scale end-user organizations 
can now control who has access to their Linux-based applica- 
tions and to manage identity information across heterogeneous 
networks of Linux, Unix, and Microsoft Windows systems from 
a central point. 

For more information, visit: http:-/Avww.cosuser.com 


Astaro Ships Version 5 of Astaro 
Security Linux 


Astaro Corp announced availability of Version 5 of its Astaro 
Security Linux, which now includes intrusion protection and added 
virus protection for HTTP and FTP. According to the company, 
Version 5 will also stop viruses in password-protected zip attach- 
ments. Available immediately from Astaro solution partners, 
licenses start at $390 for 10 users. 

For more information, visit: http://www.astaro.com 


Metapa Unveils Cluster DataBase 1.0 

Metapa Inc. announced their Cluster DataBase (CDB) 1.0. 
According to the company, it is a new Linux database clustering 
platform “purpose-built” for business intelligence. CDB is an 
alternative to SMP-based data warehousing, enabling companies 
to deploy large databases on inexpensive clusters of Linux or 
Solaris machines. 

For more information, visit: http:/Avwww.metapa.com 
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In business since 1989, ASA builds 
custom computing and storage solutions 
compatible with open-source OS (BSD, Linux) and 
also Solaris and Windows. ASA has always enjoyed strong 
financial stability and a loyal customer base. 


KA 


ASA --- Custom Servers and Storage 


www.asacomputers.com ® 866-382-5263 
2354 Calle Del Mundo, Santa Clara, CA 95054 


For details/inquiries/customization email sales@asacomputers.com 


All systems are pre-loaded with any Linux/BSD version/distribution of your choice. On-site warranty, cross-ship options available. 


MINI SUPER For Clusters 


1U 14" Depth 
1 of 2 Intel® Xeon™ processors 2.4 GHz $1,249 
Serial, VGA, USB 2.0, Mouse, Keyboard 
All ports Front Accessible 

1 x 10/100, 1 x Gigabit LAN 

512 MB DDR ECC (Max 8 GB) 

Options: CD, Floppy 


ES MSS GS DE HE ES SB EET) 
NO-FRILLS STORAGE SERVER 


$12.199 6 TB IDE/SATA storage in 5U ! 

: Dual Intel® Xeon™ processors 2.4 GHz 

512 MB DDR ECC (Max 8 GB) 

Dual Gigabit LAN, CD 

Options: SCSI Drives, Firewire, DVD+RW, 
CDRW, 64-bit OS configuration, Additional LAN, 
Floppy, Fiber Gigabit 


8 HOT-SWAP BAYS IN 2U 


1 of 2 Intel® Xeon™ processors 2.4 GHz 

512 MB DDR ECC (Max 8 GB) 

1 x 40 GB IDE 

1 x 10/100, 1 x Gigabit LAN 

Options: SATA, SCSI, Redundant Power Supply, 
Raid 0,1,5, CD, Floppy 


‘SlawServers Kill. 


, OSOL Blade Sefvers 
8UUp to 20 AMD™. Opteron™ 64bit CPU 

Upto 160GB.DDR 400MHZ.ECC- REG RAM 

Optional Ulira320SCSLRAID. controllers 

Multiple othePconfigurations available 


QSOL'1.U'Serieés Servers 

4U'Rackmount Server 

DuatAMD™ Opteron™ 64bit CPU 

Upto 16GB DDR-400MHz ECC-RAM 

~ OptidnatUltra.320 SCSI RAID €ontrollers 
Upto 4.— 146GB 10;000RPMSCSI Drives with RAID 
Up {o.4—250GB:7 200RPMS-ATA Drives-with RAID 


QSOE 2U-5U Series Servers 
2U to 5U Rackmount RAID Servers 
, ual onQuad AMD™ Opteron™ 64bit CPU 
. "Upto 32GB DDR 400MHz ECC REG RAM 
. Optional Ulira 320 SCSI RAID controllers. 
ee @ ¥ Up to 16 - 250GB #200RPM IDE Drives with RAID 
“Upto 16 - 146GBaQ,OG0RPMSCSI'Drives with RAID 
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AMD 1 >. 
& Opteron QsoL_L.com 


Server Appliances 


e AMD delivers high-performance server solutions for today's enterprise 


community 1.800.933.7510 


¢ The AMD Opteron processor minimizes integration complexity by giving 


you simultaneous high-performance 32- and 64-bit computing. WWW q SO | Cc re) mi! 
e Provides lower total cost of ownership helping to ensure long-term IT 
investment protection. ©2003 QSOL.COM All rights reserved. The QSOL.COM logo is a trademark of 9SOL.COM, 


